Sep 09 20:09:20.824693 ip-10-0-132-68 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 20:09:20.824708 ip-10-0-132-68 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 20:09:20.824718 ip-10-0-132-68 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 20:09:20.825028 ip-10-0-132-68 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 20:09:29.920978 ip-10-0-132-68 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 20:09:29.920990 ip-10-0-132-68 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 35c4f8d6bc304517a0067fea2a61823a --
Sep 09 20:11:00.958596 ip-10-0-132-68 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 20:11:01.350893 ip-10-0-132-68 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:11:01.350893 ip-10-0-132-68 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 20:11:01.350893 ip-10-0-132-68 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:11:01.350893 ip-10-0-132-68 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 20:11:01.350893 ip-10-0-132-68 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:11:01.351591 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.350952    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356129    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356145    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356149    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356152    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356155    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356158    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:11:01.356151 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356161    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356164    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356167    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356171    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356174    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356177    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356179    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356182    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356185    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356189    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356194    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356197    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356200    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356203    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356206    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356209    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356212    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356215    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356218    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:11:01.356404 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356220    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356223    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356225    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356228    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356230    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356233    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356235    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356238    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356240    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356243    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356245    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356263    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356266    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356269    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356271    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356274    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356277    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356280    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356282    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356285    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:11:01.356847 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356288    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356290    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356293    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356295    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356298    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356300    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356303    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356305    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356307    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356310    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356314    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356318    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356321    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356324    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356326    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356329    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356331    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356335    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356338    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356340    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:11:01.357330 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356342    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356345    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356347    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356350    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356352    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356355    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356358    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356360    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356363    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356365    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356367    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356370    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356373    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356376    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356378    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356380    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356383    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356385    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356389    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356391    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:11:01.357804 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356394    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356397    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356399    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356401    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356405    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356408    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356411    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356414    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356416    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356419    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356423    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356426    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356429    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.356432    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357113    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357119    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357123    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357126    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357128    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357131    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:11:01.358326 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357135    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357139    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357142    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357145    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357148    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357151    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357154    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357157    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357159    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357162    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357165    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357168    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357170    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357173    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357175    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357178    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357180    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357183    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357186    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357188    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:11:01.358802 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357191    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357193    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357196    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357199    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357202    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357204    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357206    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357209    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357211    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357214    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357218    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357220    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357223    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357226    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357229    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357231    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357233    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357236    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357239    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:11:01.359300 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357241    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357245    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357264    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357268    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357270    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357274    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357276    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357279    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357281    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357284    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357287    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357290    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357292    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357295    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357297    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357300    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357302    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357305    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357308    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:11:01.359755 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357310    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357313    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357318    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357322    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357326    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357328    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357332    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357334    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357337    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357339    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357342    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357345    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357347    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357350    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357352    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357355    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357358    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357360    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357362    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:11:01.360212 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357365    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357368    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357370    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357373    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357375    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357378    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357380    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357382    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357385    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357388    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357390    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357393    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357395    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357398    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357400    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357403    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357767    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357771    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357774    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357777    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:11:01.360683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357779    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357782    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357785    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357787    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357790    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357793    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357795    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357798    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357801    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357804    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357807    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357809    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357812    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357815    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357818    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357823    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357827    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357830    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357833    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:11:01.361147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357836    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357838    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357841    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357844    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357846    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357849    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357852    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357854    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357856    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357859    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357861    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357864    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357866    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357869    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357871    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357873    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357876    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357879    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357883    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:11:01.361673 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357886    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357889    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357892    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357894    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357897    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357899    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357902    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357905    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357908    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357910    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357913    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357915    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357918    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357920    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357923    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357925    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357928    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357930    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357933    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:11:01.362121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357935    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357938    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357941    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357944    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357947    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357950    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357952    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357955    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357958    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357961    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357964    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357967    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357969    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357971    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357976    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357979    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357981    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357983    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357986    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357988    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:11:01.362594 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357991    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357994    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357996    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.357999    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358001    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358004    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358007    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358009    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358011    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358014    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358016    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358019    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358021    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358024    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358026    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358029    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358031    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358034    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358109    2573 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358147    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358153    2573 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 20:11:01.363100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358158    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358162    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358165    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358168    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358172    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358177    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358180    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358183    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358187    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358189    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358192    2573 flags.go:64] FLAG: --cgroup-root=""
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358196    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358198    2573 flags.go:64] FLAG: --client-ca-file=""
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358202    2573 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358205    2573 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358209    2573 flags.go:64] FLAG: --cluster-domain=""
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358212    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358215    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358217    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358220    2573 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358224    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358227    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358230    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 20:11:01.363616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358233    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358236    2573 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358238    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358241    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358244    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358261    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358266    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358269    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358272    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358275    2573 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358278    2573 flags.go:64] FLAG: --enable-server="true"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358280    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358284    2573 flags.go:64] FLAG: --event-burst="100"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358287    2573 flags.go:64] FLAG: --event-qps="50"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358291    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358294    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358297    2573 flags.go:64] FLAG: --eviction-hard=""
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358300    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358303    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358305    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358310    2573 flags.go:64] FLAG: --eviction-soft=""
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358312    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358315    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358319    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358322    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 20:11:01.364157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358325    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358327    2573 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358330    2573 flags.go:64] FLAG: --feature-gates=""
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358334    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358337    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358340    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358343    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358346    2573 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358349    2573 flags.go:64] FLAG: --help="false"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358352    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358355    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358358    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358360    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358364    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358367    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358370    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358373    2573 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358375    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358378    2573 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358381    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358384    2573 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358387    2573 flags.go:64] FLAG: --kube-reserved=""
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358390    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358392    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 20:11:01.364776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358395    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358398    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358401    2573 flags.go:64] FLAG: --lock-file=""
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358405    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358407    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358410    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358415    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358419    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358421    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358424    2573 flags.go:64] FLAG: --logging-format="text"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358427    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358430    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358433    2573 flags.go:64] FLAG: --manifest-url=""
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358435    2573 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358439    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358442    2573 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358446    2573 flags.go:64] FLAG: --max-pods="110"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358449    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358452    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358455    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358458    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358460    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358463    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358466    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358474    2573 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 20:11:01.365346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358477    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358480    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358483    2573 flags.go:64] FLAG: --pod-cidr=""
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358486    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358491    2573 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358493    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358496    2573 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358499    2573 flags.go:64] FLAG: --port="10250"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358502    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358505    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d99cdd94b8484515"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358508    2573 flags.go:64] FLAG: --qos-reserved=""
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358512    2573 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358515    2573 flags.go:64] FLAG: --register-node="true"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358518    2573 flags.go:64] FLAG: --register-with-taints=""
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358521    2573 flags.go:64] FLAG: --registry-burst="10"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358525    2573 flags.go:64] FLAG: --registry-qps="5"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358527    2573 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358530    2573 flags.go:64] FLAG: --reserved-memory=""
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358534    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358537    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358539    2573 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358542    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358545    2573 flags.go:64] FLAG: --runonce="false"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358548    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358550    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 20:11:01.365932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358553    2573 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358556    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358559    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358562    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358565    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358567    2573 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358570    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358573    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358576    2573 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358579    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358581    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358584    2573 flags.go:64] FLAG: --system-cgroups=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358587    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358592    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358594    2573 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358597    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358601    2573 flags.go:64] FLAG: --tls-min-version=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358604    2573 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358608    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358611    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358613    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358616    2573 flags.go:64] FLAG: --v="2"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358619    2573 flags.go:64] FLAG: --version="false"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358624    2573 flags.go:64] FLAG: --vmodule=""
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358627    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 20:11:01.366564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.358630    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358745    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358749    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358752    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358755    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358758    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358760    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358763    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358765    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358768    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358771    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358773    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358775    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358778    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358780    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358783    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358785    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358787    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358790    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:11:01.367171 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358792    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358795    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358798    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358800    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358803    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358805    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358809    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358812    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358814    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358818    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358821    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358824    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358827    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358829    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358832    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358834    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358837    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358839    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358842    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:11:01.367641 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358846    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358849    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358852    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358855    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358859    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358862    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358865    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358867    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358870    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358872    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358875    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358878    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358880    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358883    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358885    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358888    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358890    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358893    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358895    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358899    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:11:01.368102 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358901    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358904    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358907    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358911    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358913    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358915    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358918    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358921    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358923    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358925    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358929    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358933    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358936    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358938    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358941    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358943    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358945    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358948    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358951    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358953    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:11:01.368678 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358955    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358958    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358960    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358962    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358965    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358967    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358969    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358972    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358974    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358976    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358979    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358982    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358985    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358987    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358991    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358994    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358996    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.358998    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.359001    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.359004    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:11:01.369155 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.359006    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.359009    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.359576    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.366205    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.366221    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.366971    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.366978    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367035    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367041    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367044    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367047    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367050    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367052    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367055    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367059    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:11:01.369683 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367061    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367064    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367066    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367069    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367071    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367074    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367076    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367079    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367082    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367084    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367087    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367089    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367092    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367095    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367097    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367101    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367105    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367108    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367111    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367114    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:11:01.370039 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367117    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367120    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367122    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367125    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367127    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367129    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367132    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367134    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367137    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367139    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367141    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367144    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367146    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367149    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367151    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367154    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367156    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367159    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367162    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367164    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:11:01.370542 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367167    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367169    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367172    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367174    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367177    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367180    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367183    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367185    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367188    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367191    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367194    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367196    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367199    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367201    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367204    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367207    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367209    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367211    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367214    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367216    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367219    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:11:01.371011 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367222    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367224    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367226    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367229    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367231    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367234    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367236    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367240    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367244    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367260    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367264    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367267    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367269    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367272    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367275    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367278    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367281    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367284    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:11:01.371521 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367288    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367291    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367294    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367297    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367300    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367303    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367306    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367309    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367313    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367316    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367319    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367322    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.367327    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367431    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367435    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:11:01.371953 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367438    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367441    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367443    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367446    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367449    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367452    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367454    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367457    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367459    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367461    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367464    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367466    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367469    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367472    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367474    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367477    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367480    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367482    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367485    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367487    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:11:01.372319 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367491    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367494    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367497    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367500    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367503    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367506    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367509    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367511    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367514    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367517    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367519    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367522    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367524    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367526    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367529    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367531    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367534    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367536    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367539    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367541    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:11:01.372798 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367543    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367546    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367548    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367551    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367554    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367558    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367561    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367564    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367567    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367569    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367571    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367574    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367576    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367579    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367581    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367583    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367586    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367588    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367590    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:11:01.373353 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367593    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367595    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367598    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367600    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367604    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367607    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367610    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367613    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367616    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367618    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367620    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367623    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367625    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367627    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367630    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367632    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367635    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367637    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367640    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:11:01.373806 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367642    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367645    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367647    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367650    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367653    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367655    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367658    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367660    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367662    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367665    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367667    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367670    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367672    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367675    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367678    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367680    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367683    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367685    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:11:01.374264 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:01.367688    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.367693    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.367802    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.369690    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.371123    2573 server.go:1019] "Starting client certificate rotation"
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.371220    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.371270    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:11:01.374685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.373416    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:11:01.375263 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.375226    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:11:01.383459 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.383444    2573 log.go:25] "Validated CRI v1 runtime API"
Sep 09 20:11:01.386867 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.386850    2573 log.go:25] "Validated CRI v1 image API"
Sep 09 20:11:01.388742 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.388727    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 20:11:01.391751 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.391733    2573 fs.go:135] Filesystem UUIDs: map[64de75ee-afd4-44e7-89d8-af5e35573cb8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cdad2f70-e545-4eba-9251-7393b27e83e3:/dev/nvme0n1p4]
Sep 09 20:11:01.391819 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.391750    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 20:11:01.396567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.396455    2573 manager.go:216] Machine: {Timestamp:2026-09-09 20:11:01.39515104 +0000 UTC m=+0.326234349 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200470 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22dfeb6414e40bb632842cd297e68e SystemUUID:ec22dfeb-6414-e40b-b632-842cd297e68e BootID:35c4f8d6-bc30-4517-a006-7fea2a61823a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:23:90:4a:4a:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:23:90:4a:4a:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:b0:f3:ca:47:5e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 20:11:01.396567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.396555    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 20:11:01.396709 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.396627    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 20:11:01.397542 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.397525    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:11:01.399801 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.399776    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 20:11:01.399953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.399806    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 20:11:01.400987 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.400975    2573 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 20:11:01.401042 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.400994    2573 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 20:11:01.401042 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.401005    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:11:01.401619 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.401608    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:11:01.402394 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.402383    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 20:11:01.405218 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405205    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:11:01.405356 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405346    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 20:11:01.405420 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405382    2573 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 20:11:01.405420 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405395    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 20:11:01.405420 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405408    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 20:11:01.405535 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405422    2573 kubelet.go:398] "Adding apiserver pod source"
Sep 09 20:11:01.405535 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405432    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 20:11:01.405788 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405771    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:11:01.405869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.405795    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:11:01.408169 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.408135    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 20:11:01.410648 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.410631    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 20:11:01.410710 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.410663    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 20:11:01.411936 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411925    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 20:11:01.411986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411940    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 20:11:01.411986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411946    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 20:11:01.411986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411952    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 20:11:01.411986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411961    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 20:11:01.411986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411966    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 20:11:01.411986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411982    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 20:11:01.412140 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.411990    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 20:11:01.412140 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.412001    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 20:11:01.412140 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.412007    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 20:11:01.412140 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.412015    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 20:11:01.412140 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.412020    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 20:11:01.412942 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.412929    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 20:11:01.412942 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.412943    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 20:11:01.414532 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.414512    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 20:11:01.414532 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.414512    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 20:11:01.416781 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.416768    2573 server.go:1268] "Started kubelet"
Sep 09 20:11:01.416886 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.416853    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 20:11:01.421160 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.416841    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 20:11:01.421233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.421222    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 20:11:01.421582 ip-10-0-132-68 systemd[1]: Started Kubernetes Kubelet.
Sep 09 20:11:01.422889 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.421584    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 20:11:01.422889 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.421935    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-htvnq"
Sep 09 20:11:01.423455 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.423438    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 20:11:01.424015 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.423998    2573 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 20:11:01.425003 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.424965    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-htvnq"
Sep 09 20:11:01.428055 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.428035    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 20:11:01.430168 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.429167    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-68.ec2.internal.18d3c012345dd43d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-68.ec2.internal,UID:ip-10-0-132-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-68.ec2.internal,},FirstTimestamp:2026-09-09 20:11:01.416727613 +0000 UTC m=+0.347810921,LastTimestamp:2026-09-09 20:11:01.416727613 +0000 UTC m=+0.347810921,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-68.ec2.internal,}"
Sep 09 20:11:01.432178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.432162    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 20:11:01.432884 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.432870    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 20:11:01.432973 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.432959    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.433671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433655    2573 factory.go:55] Registering systemd factory
Sep 09 20:11:01.433753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433674    2573 factory.go:223] Registration of the systemd container factory successfully
Sep 09 20:11:01.433753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433674    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 20:11:01.433753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433677    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 20:11:01.433753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433699    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 20:11:01.433887 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433769    2573 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 20:11:01.433887 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433776    2573 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 20:11:01.433887 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433883    2573 factory.go:153] Registering CRI-O factory
Sep 09 20:11:01.433962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433897    2573 factory.go:223] Registration of the crio container factory successfully
Sep 09 20:11:01.433962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433946    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 20:11:01.433962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433960    2573 factory.go:103] Registering Raw factory
Sep 09 20:11:01.434041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.433973    2573 manager.go:1196] Started watching for new ooms in manager
Sep 09 20:11:01.434295 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.434283    2573 manager.go:349] Starting recovery of all containers
Sep 09 20:11:01.435917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.435897    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:01.443152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.443137    2573 manager.go:354] Recovery completed
Sep 09 20:11:01.443629 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.443608    2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-68.ec2.internal\" not found" node="ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.450093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.450080    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.451842 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.451827    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.451926 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.451856    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.451926 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.451868    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.452390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.452378    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 20:11:01.452390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.452389    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.452487 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.452389    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 20:11:01.452487 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.452444    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:11:01.453012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.452999    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.453062 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.453018    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.453062 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.453028    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.453541 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.453529    2573 policy_none.go:49] "None policy: Start"
Sep 09 20:11:01.453584 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.453543    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 20:11:01.453584 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.453552    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 20:11:01.454487 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.454475    2573 policy_none.go:47] "Start"
Sep 09 20:11:01.477970 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.477955    2573 manager.go:337] "Starting Device Plugin manager"
Sep 09 20:11:01.478048 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.477987    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 20:11:01.478048 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.477995    2573 server.go:87] "Starting device plugin registration server"
Sep 09 20:11:01.478283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.478268    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 20:11:01.478380 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.478284    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 20:11:01.478440 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.478409    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 20:11:01.478495 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.478488    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 20:11:01.478539 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.478496    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 20:11:01.478908 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.478890    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 20:11:01.478987 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.478931    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.578823    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.580081    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.580105    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.580115    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.580140    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.591709    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.595073 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.591727    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-68.ec2.internal\": node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.609459 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.609397    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.634456 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.634429    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 20:11:01.635601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.635585    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 20:11:01.635684 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.635625    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 20:11:01.635684 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.635650    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 20:11:01.635759 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.635687    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 20:11:01.637697 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.637681    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:01.710308 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.710287    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.736671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.736651    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"]
Sep 09 20:11:01.736751 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.736724    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.737584 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.737569    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.737646 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.737598    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.737646 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.737608    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.739561 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.739548    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.739689 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.739676    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.739746 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.739709    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.740500 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.740476    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.740500 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.740489    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.740500 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.740501    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.740659 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.740513    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.740659 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.740514    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.740659 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.740526    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.742546 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.742529    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.742592 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.742565    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:11:01.743235 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.743216    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:11:01.743316 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.743267    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:11:01.743316 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.743283    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:11:01.773831 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.773816    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-68.ec2.internal\" not found" node="ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.776900 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.776885    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-68.ec2.internal\" not found" node="ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.810618 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.810602    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.836583 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.836563    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/842c5cd9014543c0f0425e48e33f68fb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal\" (UID: \"842c5cd9014543c0f0425e48e33f68fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.836652 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.836587    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/842c5cd9014543c0f0425e48e33f68fb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal\" (UID: \"842c5cd9014543c0f0425e48e33f68fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.836652 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.836605    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80384244c83dfd7f47c7ded897b4c708-config\") pod \"kube-apiserver-proxy-ip-10-0-132-68.ec2.internal\" (UID: \"80384244c83dfd7f47c7ded897b4c708\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.910977 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:01.910934    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:01.937275 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.937245    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/842c5cd9014543c0f0425e48e33f68fb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal\" (UID: \"842c5cd9014543c0f0425e48e33f68fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.937343 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.937284    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/842c5cd9014543c0f0425e48e33f68fb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal\" (UID: \"842c5cd9014543c0f0425e48e33f68fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.937343 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.937307    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80384244c83dfd7f47c7ded897b4c708-config\") pod \"kube-apiserver-proxy-ip-10-0-132-68.ec2.internal\" (UID: \"80384244c83dfd7f47c7ded897b4c708\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.937430 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.937363    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/842c5cd9014543c0f0425e48e33f68fb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal\" (UID: \"842c5cd9014543c0f0425e48e33f68fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.937430 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.937364    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/842c5cd9014543c0f0425e48e33f68fb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal\" (UID: \"842c5cd9014543c0f0425e48e33f68fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:01.937430 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:01.937364    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80384244c83dfd7f47c7ded897b4c708-config\") pod \"kube-apiserver-proxy-ip-10-0-132-68.ec2.internal\" (UID: \"80384244c83dfd7f47c7ded897b4c708\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:02.011414 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.011388    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:02.075894 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.075871    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:02.080174 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.080160    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:02.112445 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.112424    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:02.215824 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.215762    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-68.ec2.internal\" not found"
Sep 09 20:11:02.283403 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.283378    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:02.334002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.333979    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:02.346307 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.346290    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:11:02.348343 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.348317    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal"
Sep 09 20:11:02.355776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.355758    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:11:02.370519 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.370498    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 20:11:02.370614 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.370598    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:11:02.370673 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.370630    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:11:02.370724 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.370667    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:11:02.406035 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.406011    2573 apiserver.go:52] "Watching apiserver"
Sep 09 20:11:02.425553 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.425530    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 20:11:02.426501 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.426480    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vhb7f","openshift-network-diagnostics/network-check-target-59sxw","kube-system/global-pull-secret-syncer-h49zk","openshift-dns/node-resolver-jfvgh","openshift-image-registry/node-ca-xf98s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal","openshift-multus/network-metrics-daemon-5ghhl","openshift-network-operator/iptables-alerter-pnzfv","openshift-ovn-kubernetes/ovnkube-node-2frcb","kube-system/konnectivity-agent-9jmtd","kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd","openshift-cluster-node-tuning-operator/tuned-vjvhd","openshift-multus/multus-7l2m7"]
Sep 09 20:11:02.428867 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.428851    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.430328 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.430294    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:06:01 +0000 UTC" deadline="2028-06-05 15:35:14.38662961 +0000 UTC"
Sep 09 20:11:02.430328 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.430325    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15235h24m11.95630736s"
Sep 09 20:11:02.430819 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.430805    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:02.430896 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.430875    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:02.431133 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.431115    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.431400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.431384    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 20:11:02.431460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.431421    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 20:11:02.431778 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.431756    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.431871 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.431839    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 20:11:02.431871 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.431840    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dn758\""
Sep 09 20:11:02.432306 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.432267    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 20:11:02.433443 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.433274    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.433443 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.433358    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:02.435602 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.435573    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.438046 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.438023    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.438152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.438103    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.438152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.438136    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.438339 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.438157    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wt9jn\""
Sep 09 20:11:02.440105 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440085    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-system-cni-dir\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440131    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:02.440178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440173    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41cf389f-73c5-4623-ba64-c2efc1782d75-kubelet-config\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440175    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440228    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.440252    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440281    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aeecf69a-3522-49fa-924f-3208e1e1af06-hosts-file\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440307    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zfc5\" (UniqueName: \"kubernetes.io/projected/aeecf69a-3522-49fa-924f-3208e1e1af06-kube-api-access-4zfc5\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440331    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cni-binary-copy\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440352    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440375    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41cf389f-73c5-4623-ba64-c2efc1782d75-dbus\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.440682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440399    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440457    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrmr8\" (UniqueName: \"kubernetes.io/projected/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-kube-api-access-qrmr8\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440520    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeecf69a-3522-49fa-924f-3208e1e1af06-tmp-dir\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.440682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440584    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cnibin\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.440919 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440768    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 20:11:02.440919 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440813    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9vskd\""
Sep 09 20:11:02.440919 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.440768    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.441114 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.441090    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.442509 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.442492    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.444586 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.444566    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-chlqn\""
Sep 09 20:11:02.444940 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.444926    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.445028 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.444959    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 20:11:02.445028 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.444971    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.445138 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.445045    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.447264 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.447242    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 20:11:02.447387 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.447246    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.448308 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.448289    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 20:11:02.448308 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.448305    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 20:11:02.448454 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.448311    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.448454 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.448338    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 20:11:02.448454 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.448370    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.448454 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.448311    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m6f54\""
Sep 09 20:11:02.449631 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.449612    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 20:11:02.449631 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.449624    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2fwlt\""
Sep 09 20:11:02.449768 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.449681    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.449826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.449617    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 20:11:02.451075 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.451056    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:11:02.451959 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.451936    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 20:11:02.452240 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.452215    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.452640 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.452461    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b946f\""
Sep 09 20:11:02.452709 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.452675    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.453161 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.453143    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.454848 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.454830    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.455480 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.455454    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:11:02.455590 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.455523    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 20:11:02.455939 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.455923    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7kqgk\""
Sep 09 20:11:02.457311 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.457293    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 20:11:02.457815 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.457797    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dbjdx\""
Sep 09 20:11:02.471393 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.471348    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7mvx4"
Sep 09 20:11:02.479854 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.479832    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7mvx4"
Sep 09 20:11:02.531026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.531009    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:02.534620 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.534594    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 20:11:02.540810 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540784    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-kubernetes\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.540904 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540828    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-cni-bin\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.540904 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540861    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-daemon-config\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.540904 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540892    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg2rl\" (UniqueName: \"kubernetes.io/projected/e585404d-4349-472e-ba0d-3f8bd42682d4-kube-api-access-bg2rl\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.541036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.541036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540966    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.540991    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovnkube-config\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541015    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-env-overrides\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541041    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-socket-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541063    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-lib-modules\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541087    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-socket-dir-parent\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c25adba-3b18-4022-8c5d-66c3ac0d9511-iptables-alerter-script\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541149    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41cf389f-73c5-4623-ba64-c2efc1782d75-kubelet-config\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541179    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.541221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541211    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfc5\" (UniqueName: \"kubernetes.io/projected/aeecf69a-3522-49fa-924f-3208e1e1af06-kube-api-access-4zfc5\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.541533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541244    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41cf389f-73c5-4623-ba64-c2efc1782d75-kubelet-config\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.541533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cni-binary-copy\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.541533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541303    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrmr8\" (UniqueName: \"kubernetes.io/projected/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-kube-api-access-qrmr8\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.541533 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.541388    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:02.541533 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.541487    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret podName:41cf389f-73c5-4623-ba64-c2efc1782d75 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:03.041442055 +0000 UTC m=+1.972525355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret") pod "global-pull-secret-syncer-h49zk" (UID: "41cf389f-73c5-4623-ba64-c2efc1782d75") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:02.541533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541513    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv698\" (UniqueName: \"kubernetes.io/projected/80a080ef-623f-4b93-be6c-2b4355b2afc2-kube-api-access-jv698\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541544    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-kubelet\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541571    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-log-socket\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541600    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41cf389f-73c5-4623-ba64-c2efc1782d75-dbus\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541629    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541656    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-node-log\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541681    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr67x\" (UniqueName: \"kubernetes.io/projected/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-kube-api-access-nr67x\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541720    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41cf389f-73c5-4623-ba64-c2efc1782d75-dbus\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541728    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9063d1b5-2af6-4334-a199-9addeb600283-konnectivity-ca\") pod \"konnectivity-agent-9jmtd\" (UID: \"9063d1b5-2af6-4334-a199-9addeb600283\") " pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541765    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-modprobe-d\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541792    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysconfig\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.541818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541817    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-run\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541841    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-registration-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541878    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-device-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541902    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-sys-fs\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541927    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrxms\" (UniqueName: \"kubernetes.io/projected/8d5b378c-63fd-421d-a17d-389aaf56f437-kube-api-access-lrxms\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.541971    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e585404d-4349-472e-ba0d-3f8bd42682d4-cni-binary-copy\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542008    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-kubelet\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542044    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-multus-certs\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e8e60aba-2480-4462-ac8b-15a1a89661fd-serviceca\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542097    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-cnibin\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-netns\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542145    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-hostroot\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542175    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4qjn\" (UniqueName: \"kubernetes.io/projected/e8e60aba-2480-4462-ac8b-15a1a89661fd-kube-api-access-x4qjn\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542204    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c25adba-3b18-4022-8c5d-66c3ac0d9511-host-slash\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542230    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj9lt\" (UniqueName: \"kubernetes.io/projected/7c25adba-3b18-4022-8c5d-66c3ac0d9511-kube-api-access-nj9lt\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542280    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542318    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-run-netns\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.542317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542341    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-sys\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542368    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-k8s-cni-cncf-io\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542391    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-var-lib-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovn-node-metrics-cert\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542437    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-tuned\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542449    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542460    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-cni-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542495    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-cni-multus\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542523    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-systemd-units\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542546    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-slash\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542571    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-system-cni-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542608    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-etc-kubernetes\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542635    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-system-cni-dir\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542661    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542688    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-etc-selinux\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542714    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cni-binary-copy\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542711    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-host\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542713    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542770    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9063d1b5-2af6-4334-a199-9addeb600283-agent-certs\") pod \"konnectivity-agent-9jmtd\" (UID: \"9063d1b5-2af6-4334-a199-9addeb600283\") " pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542790    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-system-cni-dir\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542807    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysctl-conf\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542827    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cnibin\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542874    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-cni-bin\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542906    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-systemd\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542928    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d5b378c-63fd-421d-a17d-389aaf56f437-tmp\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542951    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8e60aba-2480-4462-ac8b-15a1a89661fd-host\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.542981    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543005    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-etc-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543029    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-ovn\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543040    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543054    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjc9q\" (UniqueName: \"kubernetes.io/projected/7c0456c5-dd40-4707-a5ab-97f67694fe81-kube-api-access-rjc9q\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543077    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysctl-d\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543112    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aeecf69a-3522-49fa-924f-3208e1e1af06-hosts-file\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.543953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543167    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aeecf69a-3522-49fa-924f-3208e1e1af06-hosts-file\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543221    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-cnibin\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543252    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543277    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-cni-netd\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543335    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovnkube-script-lib\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543364    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543390    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-var-lib-kubelet\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-systemd\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543443    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-conf-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543471    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeecf69a-3522-49fa-924f-3208e1e1af06-tmp-dir\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543495    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.544520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.543742    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeecf69a-3522-49fa-924f-3208e1e1af06-tmp-dir\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.551707 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.551685    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 20:11:02.551805 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.551745    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:11:02.551805 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.551762    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:11:02.551805 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.551772    2573 projected.go:196] Error preparing data for projected volume kube-api-access-6mqlc for pod openshift-network-diagnostics/network-check-target-59sxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:02.551986 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.551824    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc podName:1263e31c-c43e-4325-829b-e03f458e3038 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:03.051812053 +0000 UTC m=+1.982895351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6mqlc" (UniqueName: "kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc") pod "network-check-target-59sxw" (UID: "1263e31c-c43e-4325-829b-e03f458e3038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:02.555259 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.555150    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrmr8\" (UniqueName: \"kubernetes.io/projected/4f07d8e6-1c97-4b4b-96d6-027b2f9f6195-kube-api-access-qrmr8\") pod \"multus-additional-cni-plugins-vhb7f\" (UID: \"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195\") " pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.555329 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.555300    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfc5\" (UniqueName: \"kubernetes.io/projected/aeecf69a-3522-49fa-924f-3208e1e1af06-kube-api-access-4zfc5\") pod \"node-resolver-jfvgh\" (UID: \"aeecf69a-3522-49fa-924f-3208e1e1af06\") " pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.643807 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643779    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-kubelet\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643807 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643807    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-log-socket\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643823    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-node-log\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643840    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr67x\" (UniqueName: \"kubernetes.io/projected/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-kube-api-access-nr67x\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643862    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9063d1b5-2af6-4334-a199-9addeb600283-konnectivity-ca\") pod \"konnectivity-agent-9jmtd\" (UID: \"9063d1b5-2af6-4334-a199-9addeb600283\") " pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-modprobe-d\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643889    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-log-socket\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643905    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-node-log\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643912    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysconfig\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643925    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-kubelet\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.643977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.643970    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysconfig\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644002    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-run\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644022    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-registration-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644026    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-modprobe-d\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-device-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644084    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-device-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644108    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-run\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644109    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-registration-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644125    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-sys-fs\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644144    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrxms\" (UniqueName: \"kubernetes.io/projected/8d5b378c-63fd-421d-a17d-389aaf56f437-kube-api-access-lrxms\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644153    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-sys-fs\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e585404d-4349-472e-ba0d-3f8bd42682d4-cni-binary-copy\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644191    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-kubelet\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644215    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-multus-certs\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644237    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e8e60aba-2480-4462-ac8b-15a1a89661fd-serviceca\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644274    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-cnibin\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644287    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-multus-certs\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644297    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-netns\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.644347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644289    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-kubelet\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644336    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-netns\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644342    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-cnibin\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644362    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-hostroot\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4qjn\" (UniqueName: \"kubernetes.io/projected/e8e60aba-2480-4462-ac8b-15a1a89661fd-kube-api-access-x4qjn\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644393    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-hostroot\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644461    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9063d1b5-2af6-4334-a199-9addeb600283-konnectivity-ca\") pod \"konnectivity-agent-9jmtd\" (UID: \"9063d1b5-2af6-4334-a199-9addeb600283\") " pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644493    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c25adba-3b18-4022-8c5d-66c3ac0d9511-host-slash\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj9lt\" (UniqueName: \"kubernetes.io/projected/7c25adba-3b18-4022-8c5d-66c3ac0d9511-kube-api-access-nj9lt\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644531    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-run-netns\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644546    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-sys\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644560    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-k8s-cni-cncf-io\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644568    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c25adba-3b18-4022-8c5d-66c3ac0d9511-host-slash\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-var-lib-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644597    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-run-netns\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644626    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-run-k8s-cni-cncf-io\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644631    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-sys\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644653    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovn-node-metrics-cert\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.645141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644673    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-var-lib-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644679    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-tuned\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644702    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-cni-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644719    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e585404d-4349-472e-ba0d-3f8bd42682d4-cni-binary-copy\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644725    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-cni-multus\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644734    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e8e60aba-2480-4462-ac8b-15a1a89661fd-serviceca\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644753    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-systemd-units\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644770    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-cni-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644778    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-slash\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644783    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-cni-multus\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644797    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-systemd-units\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644801    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-system-cni-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644824    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-slash\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644826    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-etc-kubernetes\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644851    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-etc-kubernetes\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644861    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644885    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-system-cni-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644923    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.646068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-etc-selinux\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644934    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-etc-selinux\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-host\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644967    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9063d1b5-2af6-4334-a199-9addeb600283-agent-certs\") pod \"konnectivity-agent-9jmtd\" (UID: \"9063d1b5-2af6-4334-a199-9addeb600283\") " pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644980    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysctl-conf\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.644995    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-cni-bin\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645014    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-systemd\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645058    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-cni-bin\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645072    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-host\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645089    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d5b378c-63fd-421d-a17d-389aaf56f437-tmp\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645115    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8e60aba-2480-4462-ac8b-15a1a89661fd-host\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645117    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-systemd\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645147    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysctl-conf\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645152    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-etc-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645183    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-ovn\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645207    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8e60aba-2480-4462-ac8b-15a1a89661fd-host\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645215    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-ovn\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645210    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjc9q\" (UniqueName: \"kubernetes.io/projected/7c0456c5-dd40-4707-a5ab-97f67694fe81-kube-api-access-rjc9q\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.647060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645186    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-etc-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645268    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysctl-d\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645295    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645314    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-cni-netd\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovnkube-script-lib\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645356    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.645412    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645430    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645434    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-cni-netd\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.645467    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:03.145448784 +0000 UTC m=+2.076532098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645487    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-var-lib-kubelet\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645513    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-systemd\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645519    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-sysctl-d\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645536    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-conf-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645619    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-var-lib-kubelet\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645670    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645696    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-kubernetes\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.647825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645737    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-cni-bin\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645742    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-systemd\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645763    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-daemon-config\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645755    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-kubernetes\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645788    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-conf-dir\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645810    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-run-openvswitch\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645830    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-host-var-lib-cni-bin\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645837    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg2rl\" (UniqueName: \"kubernetes.io/projected/e585404d-4349-472e-ba0d-3f8bd42682d4-kube-api-access-bg2rl\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645873    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645898    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovnkube-config\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.645922    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-env-overrides\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646028    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovnkube-script-lib\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646042    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646097    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-socket-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646142    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-lib-modules\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646169    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-socket-dir-parent\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646193    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c25adba-3b18-4022-8c5d-66c3ac0d9511-iptables-alerter-script\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.648386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646236    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv698\" (UniqueName: \"kubernetes.io/projected/80a080ef-623f-4b93-be6c-2b4355b2afc2-kube-api-access-jv698\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646287    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-env-overrides\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d5b378c-63fd-421d-a17d-389aaf56f437-lib-modules\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646497    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-socket-dir-parent\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646517    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7c0456c5-dd40-4707-a5ab-97f67694fe81-socket-dir\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646557    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e585404d-4349-472e-ba0d-3f8bd42682d4-multus-daemon-config\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646598    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovnkube-config\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.646902    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c25adba-3b18-4022-8c5d-66c3ac0d9511-iptables-alerter-script\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.647645    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d5b378c-63fd-421d-a17d-389aaf56f437-tmp\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.647696    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-ovn-node-metrics-cert\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.647768    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9063d1b5-2af6-4334-a199-9addeb600283-agent-certs\") pod \"konnectivity-agent-9jmtd\" (UID: \"9063d1b5-2af6-4334-a199-9addeb600283\") " pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.648897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.648284    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8d5b378c-63fd-421d-a17d-389aaf56f437-etc-tuned\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.656101 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.656019    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr67x\" (UniqueName: \"kubernetes.io/projected/98cddd01-8d4d-45e7-86d0-f8edb8d38c1a-kube-api-access-nr67x\") pod \"ovnkube-node-2frcb\" (UID: \"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.657607 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.657564    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg2rl\" (UniqueName: \"kubernetes.io/projected/e585404d-4349-472e-ba0d-3f8bd42682d4-kube-api-access-bg2rl\") pod \"multus-7l2m7\" (UID: \"e585404d-4349-472e-ba0d-3f8bd42682d4\") " pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.657775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.657754    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrxms\" (UniqueName: \"kubernetes.io/projected/8d5b378c-63fd-421d-a17d-389aaf56f437-kube-api-access-lrxms\") pod \"tuned-vjvhd\" (UID: \"8d5b378c-63fd-421d-a17d-389aaf56f437\") " pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.658185 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.658163    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv698\" (UniqueName: \"kubernetes.io/projected/80a080ef-623f-4b93-be6c-2b4355b2afc2-kube-api-access-jv698\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:02.658262 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.658237    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4qjn\" (UniqueName: \"kubernetes.io/projected/e8e60aba-2480-4462-ac8b-15a1a89661fd-kube-api-access-x4qjn\") pod \"node-ca-xf98s\" (UID: \"e8e60aba-2480-4462-ac8b-15a1a89661fd\") " pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.658262 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.658249    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjc9q\" (UniqueName: \"kubernetes.io/projected/7c0456c5-dd40-4707-a5ab-97f67694fe81-kube-api-access-rjc9q\") pod \"aws-ebs-csi-driver-node-sdmqd\" (UID: \"7c0456c5-dd40-4707-a5ab-97f67694fe81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.658496 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.658479    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj9lt\" (UniqueName: \"kubernetes.io/projected/7c25adba-3b18-4022-8c5d-66c3ac0d9511-kube-api-access-nj9lt\") pod \"iptables-alerter-pnzfv\" (UID: \"7c25adba-3b18-4022-8c5d-66c3ac0d9511\") " pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.693613 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.693576    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842c5cd9014543c0f0425e48e33f68fb.slice/crio-c293576aa5e34ef374ed123dd55e6d5a57e14ca79e6689df7d846e89b7592e69 WatchSource:0}: Error finding container c293576aa5e34ef374ed123dd55e6d5a57e14ca79e6689df7d846e89b7592e69: Status 404 returned error can't find the container with id c293576aa5e34ef374ed123dd55e6d5a57e14ca79e6689df7d846e89b7592e69
Sep 09 20:11:02.693925 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.693906    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80384244c83dfd7f47c7ded897b4c708.slice/crio-7ae311df5559931f733dd175d085cbf4d55ca71be1a3ec66c63bdebe5acb634a WatchSource:0}: Error finding container 7ae311df5559931f733dd175d085cbf4d55ca71be1a3ec66c63bdebe5acb634a: Status 404 returned error can't find the container with id 7ae311df5559931f733dd175d085cbf4d55ca71be1a3ec66c63bdebe5acb634a
Sep 09 20:11:02.698305 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.698291    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:11:02.755328 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.755280    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vhb7f"
Sep 09 20:11:02.764902 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.764880    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f07d8e6_1c97_4b4b_96d6_027b2f9f6195.slice/crio-a2b4bf5da99601a502cdb448196f34eb2459e05f0dbfe63ac0c8b8f4aa0655be WatchSource:0}: Error finding container a2b4bf5da99601a502cdb448196f34eb2459e05f0dbfe63ac0c8b8f4aa0655be: Status 404 returned error can't find the container with id a2b4bf5da99601a502cdb448196f34eb2459e05f0dbfe63ac0c8b8f4aa0655be
Sep 09 20:11:02.784001 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.783968    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jfvgh"
Sep 09 20:11:02.791651 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.791632    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xf98s"
Sep 09 20:11:02.791973 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.791951    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeecf69a_3522_49fa_924f_3208e1e1af06.slice/crio-caac1352c9350d562837bf12dc9888688e5fd2e5cd1c13ef4e5f7f63b910d38d WatchSource:0}: Error finding container caac1352c9350d562837bf12dc9888688e5fd2e5cd1c13ef4e5f7f63b910d38d: Status 404 returned error can't find the container with id caac1352c9350d562837bf12dc9888688e5fd2e5cd1c13ef4e5f7f63b910d38d
Sep 09 20:11:02.800293 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.800274    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e60aba_2480_4462_ac8b_15a1a89661fd.slice/crio-f51b30b0928968ea863970aa87f860e942c0ca976924dedba7a6388b2708c61d WatchSource:0}: Error finding container f51b30b0928968ea863970aa87f860e942c0ca976924dedba7a6388b2708c61d: Status 404 returned error can't find the container with id f51b30b0928968ea863970aa87f860e942c0ca976924dedba7a6388b2708c61d
Sep 09 20:11:02.804921 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.804904    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pnzfv"
Sep 09 20:11:02.813005 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.812988    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c25adba_3b18_4022_8c5d_66c3ac0d9511.slice/crio-18fc3d27b8c23af0023a777ff2a78582eb2837eecfaf3e7f9c9ca21788418095 WatchSource:0}: Error finding container 18fc3d27b8c23af0023a777ff2a78582eb2837eecfaf3e7f9c9ca21788418095: Status 404 returned error can't find the container with id 18fc3d27b8c23af0023a777ff2a78582eb2837eecfaf3e7f9c9ca21788418095
Sep 09 20:11:02.823998 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.823976    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:02.831864 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.831844    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98cddd01_8d4d_45e7_86d0_f8edb8d38c1a.slice/crio-7f426894f91ddf6c6211c8afe2f9f74c8705b23fe55fb07b1c4e51f3b3cd941b WatchSource:0}: Error finding container 7f426894f91ddf6c6211c8afe2f9f74c8705b23fe55fb07b1c4e51f3b3cd941b: Status 404 returned error can't find the container with id 7f426894f91ddf6c6211c8afe2f9f74c8705b23fe55fb07b1c4e51f3b3cd941b
Sep 09 20:11:02.840411 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.840394    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:02.847458 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.847439    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd"
Sep 09 20:11:02.848331 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.848296    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9063d1b5_2af6_4334_a199_9addeb600283.slice/crio-85e7d7e3b7fae89b8f2a4df6917e52062e542640338924404cf68dfa6bb7a104 WatchSource:0}: Error finding container 85e7d7e3b7fae89b8f2a4df6917e52062e542640338924404cf68dfa6bb7a104: Status 404 returned error can't find the container with id 85e7d7e3b7fae89b8f2a4df6917e52062e542640338924404cf68dfa6bb7a104
Sep 09 20:11:02.856131 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.856109    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c0456c5_dd40_4707_a5ab_97f67694fe81.slice/crio-88462476123c7a57c06cfcf491fc835c5de344b3a46c9ff5404086eee1949c9e WatchSource:0}: Error finding container 88462476123c7a57c06cfcf491fc835c5de344b3a46c9ff5404086eee1949c9e: Status 404 returned error can't find the container with id 88462476123c7a57c06cfcf491fc835c5de344b3a46c9ff5404086eee1949c9e
Sep 09 20:11:02.869340 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.869316    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjvhd"
Sep 09 20:11:02.877606 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.877588    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5b378c_63fd_421d_a17d_389aaf56f437.slice/crio-2c1b4412174050017bf0ee761de0e130eaf0c97ef31f0135f33388357f4291ea WatchSource:0}: Error finding container 2c1b4412174050017bf0ee761de0e130eaf0c97ef31f0135f33388357f4291ea: Status 404 returned error can't find the container with id 2c1b4412174050017bf0ee761de0e130eaf0c97ef31f0135f33388357f4291ea
Sep 09 20:11:02.882498 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:02.882482    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7l2m7"
Sep 09 20:11:02.891121 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:02.891096    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode585404d_4349_472e_ba0d_3f8bd42682d4.slice/crio-63d4515af80df7ea886124a61c9fe6116435fe88fbb8575f52bf7bc720c4d8a7 WatchSource:0}: Error finding container 63d4515af80df7ea886124a61c9fe6116435fe88fbb8575f52bf7bc720c4d8a7: Status 404 returned error can't find the container with id 63d4515af80df7ea886124a61c9fe6116435fe88fbb8575f52bf7bc720c4d8a7
Sep 09 20:11:02.892696 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.892679    2573 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-7l2m7_openshift-multus(e585404d-4349-472e-ba0d-3f8bd42682d4): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 20:11:02.892763 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:02.892712    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-7l2m7" podUID="e585404d-4349-472e-ba0d-3f8bd42682d4"
Sep 09 20:11:03.048315 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.048235    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:03.048409 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.048382    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:03.048456 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.048435    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret podName:41cf389f-73c5-4623-ba64-c2efc1782d75 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:04.04841743 +0000 UTC m=+2.979500727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret") pod "global-pull-secret-syncer-h49zk" (UID: "41cf389f-73c5-4623-ba64-c2efc1782d75") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:03.149275 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.149231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:03.149406 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.149295    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:03.149475 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.149452    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:03.149524 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.149511    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:04.149492686 +0000 UTC m=+3.080576019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:03.149762 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.149735    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:11:03.149762 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.149763    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:11:03.149920 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.149776    2573 projected.go:196] Error preparing data for projected volume kube-api-access-6mqlc for pod openshift-network-diagnostics/network-check-target-59sxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:03.149920 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.149827    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc podName:1263e31c-c43e-4325-829b-e03f458e3038 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:04.149809853 +0000 UTC m=+3.080893169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mqlc" (UniqueName: "kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc") pod "network-check-target-59sxw" (UID: "1263e31c-c43e-4325-829b-e03f458e3038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:03.372263 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.372232    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:03.483759 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.483642    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:06:02 +0000 UTC" deadline="2028-05-15 09:10:32.807161428 +0000 UTC"
Sep 09 20:11:03.483759 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.483710    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14724h59m29.32345639s"
Sep 09 20:11:03.638293 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.637127    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:03.638293 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.637262    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:03.651823 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.651766    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7l2m7" event={"ID":"e585404d-4349-472e-ba0d-3f8bd42682d4","Type":"ContainerStarted","Data":"63d4515af80df7ea886124a61c9fe6116435fe88fbb8575f52bf7bc720c4d8a7"}
Sep 09 20:11:03.655052 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:03.654429    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-7l2m7" podUID="e585404d-4349-472e-ba0d-3f8bd42682d4"
Sep 09 20:11:03.664140 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.663989    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd" event={"ID":"7c0456c5-dd40-4707-a5ab-97f67694fe81","Type":"ContainerStarted","Data":"88462476123c7a57c06cfcf491fc835c5de344b3a46c9ff5404086eee1949c9e"}
Sep 09 20:11:03.673538 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.673511    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jmtd" event={"ID":"9063d1b5-2af6-4334-a199-9addeb600283","Type":"ContainerStarted","Data":"85e7d7e3b7fae89b8f2a4df6917e52062e542640338924404cf68dfa6bb7a104"}
Sep 09 20:11:03.687720 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.687649    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pnzfv" event={"ID":"7c25adba-3b18-4022-8c5d-66c3ac0d9511","Type":"ContainerStarted","Data":"18fc3d27b8c23af0023a777ff2a78582eb2837eecfaf3e7f9c9ca21788418095"}
Sep 09 20:11:03.697671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.697638    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xf98s" event={"ID":"e8e60aba-2480-4462-ac8b-15a1a89661fd","Type":"ContainerStarted","Data":"f51b30b0928968ea863970aa87f860e942c0ca976924dedba7a6388b2708c61d"}
Sep 09 20:11:03.704987 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.704955    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjvhd" event={"ID":"8d5b378c-63fd-421d-a17d-389aaf56f437","Type":"ContainerStarted","Data":"2c1b4412174050017bf0ee761de0e130eaf0c97ef31f0135f33388357f4291ea"}
Sep 09 20:11:03.710745 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.710719    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"7f426894f91ddf6c6211c8afe2f9f74c8705b23fe55fb07b1c4e51f3b3cd941b"}
Sep 09 20:11:03.718774 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.718746    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jfvgh" event={"ID":"aeecf69a-3522-49fa-924f-3208e1e1af06","Type":"ContainerStarted","Data":"caac1352c9350d562837bf12dc9888688e5fd2e5cd1c13ef4e5f7f63b910d38d"}
Sep 09 20:11:03.731186 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.731156    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerStarted","Data":"a2b4bf5da99601a502cdb448196f34eb2459e05f0dbfe63ac0c8b8f4aa0655be"}
Sep 09 20:11:03.753038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.753009    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal" event={"ID":"842c5cd9014543c0f0425e48e33f68fb","Type":"ContainerStarted","Data":"c293576aa5e34ef374ed123dd55e6d5a57e14ca79e6689df7d846e89b7592e69"}
Sep 09 20:11:03.761179 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.761142    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal" event={"ID":"80384244c83dfd7f47c7ded897b4c708","Type":"ContainerStarted","Data":"7ae311df5559931f733dd175d085cbf4d55ca71be1a3ec66c63bdebe5acb634a"}
Sep 09 20:11:03.763208 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:03.763185    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:04.058315 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.058228    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:04.058570 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.058440    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:04.058570 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.058504    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret podName:41cf389f-73c5-4623-ba64-c2efc1782d75 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:06.058485539 +0000 UTC m=+4.989568861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret") pod "global-pull-secret-syncer-h49zk" (UID: "41cf389f-73c5-4623-ba64-c2efc1782d75") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:04.159106 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.159067    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:04.159325 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.159165    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:04.159413 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.159382    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:04.159472 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.159448    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:06.159427962 +0000 UTC m=+5.090511265 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:04.159945 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.159924    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:11:04.160045 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.159954    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:11:04.160045 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.159968    2573 projected.go:196] Error preparing data for projected volume kube-api-access-6mqlc for pod openshift-network-diagnostics/network-check-target-59sxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:04.160045 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.160019    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc podName:1263e31c-c43e-4325-829b-e03f458e3038 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:06.160003276 +0000 UTC m=+5.091086578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mqlc" (UniqueName: "kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc") pod "network-check-target-59sxw" (UID: "1263e31c-c43e-4325-829b-e03f458e3038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:04.486797 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.486728    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:06:02 +0000 UTC" deadline="2028-05-11 13:24:57.792298041 +0000 UTC"
Sep 09 20:11:04.486797 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.486794    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14633h13m53.30550858s"
Sep 09 20:11:04.636901 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.636867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:04.637062 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.637017    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:04.637128 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.637119    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:04.637590 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:04.637208    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:04.991283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:04.991231    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:11:05.639400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:05.639365    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:05.639877 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:05.639504    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:06.074657 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:06.074620    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:06.074816 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.074784    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:06.074894 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.074848    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret podName:41cf389f-73c5-4623-ba64-c2efc1782d75 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:10.074828594 +0000 UTC m=+9.005911895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret") pod "global-pull-secret-syncer-h49zk" (UID: "41cf389f-73c5-4623-ba64-c2efc1782d75") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:06.175399 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:06.175367    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:06.175578 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:06.175414    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:06.175578 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.175569    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:06.175694 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.175597    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:11:06.175694 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.175625    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:11:06.175694 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.175639    2573 projected.go:196] Error preparing data for projected volume kube-api-access-6mqlc for pod openshift-network-diagnostics/network-check-target-59sxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:06.175694 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.175639    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:10.175620078 +0000 UTC m=+9.106703397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:06.175896 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.175705    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc podName:1263e31c-c43e-4325-829b-e03f458e3038 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:10.17568745 +0000 UTC m=+9.106770752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mqlc" (UniqueName: "kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc") pod "network-check-target-59sxw" (UID: "1263e31c-c43e-4325-829b-e03f458e3038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:06.636400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:06.636286    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:06.636544 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.636443    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:06.636879 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:06.636852    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:06.637001 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:06.636966    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:07.636929 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:07.636888    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:07.637568 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:07.637053    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:08.636941 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:08.636908    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:08.637369 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:08.637026    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:08.637369 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:08.637086    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:08.637369 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:08.637205    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:09.636732 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:09.636264    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:09.636732 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:09.636392    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:10.105727 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:10.105692    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:10.106157 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.105838    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:10.106157 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.105910    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret podName:41cf389f-73c5-4623-ba64-c2efc1782d75 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:18.105891195 +0000 UTC m=+17.036974502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret") pod "global-pull-secret-syncer-h49zk" (UID: "41cf389f-73c5-4623-ba64-c2efc1782d75") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:10.206483 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:10.206451    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:10.206614 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:10.206494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:10.206614 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.206601    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:10.206724 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.206615    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:11:10.206724 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.206634    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:11:10.206724 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.206646    2573 projected.go:196] Error preparing data for projected volume kube-api-access-6mqlc for pod openshift-network-diagnostics/network-check-target-59sxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:10.206724 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.206666    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:18.206648918 +0000 UTC m=+17.137732437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:10.206724 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.206681    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc podName:1263e31c-c43e-4325-829b-e03f458e3038 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:18.20667213 +0000 UTC m=+17.137755446 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mqlc" (UniqueName: "kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc") pod "network-check-target-59sxw" (UID: "1263e31c-c43e-4325-829b-e03f458e3038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:10.635880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:10.635850    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:10.635880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:10.635876    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:10.636111 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.635974    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:10.636168 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:10.636109    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:11.637026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:11.636964    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:11.637503 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:11.637067    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:12.636212 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:12.636178    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:12.636391 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:12.636180    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:12.636391 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:12.636310    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:12.636510 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:12.636392    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:13.636262 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:13.636228    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:13.636727 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:13.636373    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:14.636436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:14.636402    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:14.636824 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:14.636445    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:14.636824 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:14.636525    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:14.636824 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:14.636650    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:15.636419 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:15.636388    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:15.636564 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:15.636494    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:16.636353 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:16.636332    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:16.636442 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:16.636336    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:16.636503 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:16.636420    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:16.636556 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:16.636542    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:16.792150 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:16.791908    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal" event={"ID":"80384244c83dfd7f47c7ded897b4c708","Type":"ContainerStarted","Data":"9438c53cd34ea410856c16849e000462db08c0dd4406f645b641cde7089bffd5"}
Sep 09 20:11:16.793700 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:16.793679    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"8dc3f37c329bea3db70cb9b6c43d90987c67a3fa6daf46a36d4ed174542c8251"}
Sep 09 20:11:16.805523 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:16.805476    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-68.ec2.internal" podStartSLOduration=14.80546084 podStartE2EDuration="14.80546084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:16.804925034 +0000 UTC m=+15.736008368" watchObservedRunningTime="2026-09-09 20:11:16.80546084 +0000 UTC m=+15.736544159"
Sep 09 20:11:17.636956 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.636751    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:17.637128 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:17.637029    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:17.799011 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.798654    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjvhd" event={"ID":"8d5b378c-63fd-421d-a17d-389aaf56f437","Type":"ContainerStarted","Data":"f32b4ba85b488c22d54329c4e8d1c99d0d9140f364c680396450762d746d47a4"}
Sep 09 20:11:17.804905 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.804872    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:11:17.805935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.805396    2573 generic.go:356] "Generic (PLEG): container finished" podID="98cddd01-8d4d-45e7-86d0-f8edb8d38c1a" containerID="c4c6a0cf32e83f292993208ac035204976c2cdbd9d320d915e052f1c2d99641b" exitCode=1
Sep 09 20:11:17.805935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.805467    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"227bdf6f4bc5f5520d708799936b561090117b000d77e24bff182cb3e61ac378"}
Sep 09 20:11:17.805935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.805569    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"99a739dca50e08c6f4cada271b75becc6a12b293443858a02649c4a4ea7a9603"}
Sep 09 20:11:17.805935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.805625    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"cadaca0108b3ae7a0b178918b81511873820fb0d31cd7c6237e87fe8f63afa86"}
Sep 09 20:11:17.805935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.805679    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"1ee3876cc0010c57acdd3f9db3b660cddff2b363898d909b889b179a6ae9d52a"}
Sep 09 20:11:17.805935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.805733    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerDied","Data":"c4c6a0cf32e83f292993208ac035204976c2cdbd9d320d915e052f1c2d99641b"}
Sep 09 20:11:17.815282 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.814993    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jmtd" event={"ID":"9063d1b5-2af6-4334-a199-9addeb600283","Type":"ContainerStarted","Data":"2aa4ef96cefa203ab91f6aa58c0e65b5e18dd50639801b7db44fc2cda900195c"}
Sep 09 20:11:17.816891 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.816844    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vjvhd" podStartSLOduration=1.7735926119999998 podStartE2EDuration="15.816827089s" totalImagesPullingTime="14.043234477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:17.816622839 +0000 UTC m=+16.747706160" watchObservedRunningTime="2026-09-09 20:11:17.816827089 +0000 UTC m=+16.747910413"
Sep 09 20:11:17.856806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.856761    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9jmtd" podStartSLOduration=2.085091124 podStartE2EDuration="15.856742383s" totalImagesPullingTime="13.771651259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:17.856573205 +0000 UTC m=+16.787656525" watchObservedRunningTime="2026-09-09 20:11:17.856742383 +0000 UTC m=+16.787825701"
Sep 09 20:11:17.873320 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.873275    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jfvgh" podStartSLOduration=2.035118056 podStartE2EDuration="15.873245067s" totalImagesPullingTime="13.838127011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:17.872785794 +0000 UTC m=+16.803869104" watchObservedRunningTime="2026-09-09 20:11:17.873245067 +0000 UTC m=+16.804328386"
Sep 09 20:11:17.902818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:17.902734    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xf98s" podStartSLOduration=2.442215663 podStartE2EDuration="15.902719372s" totalImagesPullingTime="13.460503709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:17.902563334 +0000 UTC m=+16.833646653" watchObservedRunningTime="2026-09-09 20:11:17.902719372 +0000 UTC m=+16.833802690"
Sep 09 20:11:18.164333 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.164191    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:18.164500 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.164332    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:18.164500 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.164394    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret podName:41cf389f-73c5-4623-ba64-c2efc1782d75 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:34.164372813 +0000 UTC m=+33.095456110 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret") pod "global-pull-secret-syncer-h49zk" (UID: "41cf389f-73c5-4623-ba64-c2efc1782d75") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:11:18.264949 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.264901    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:18.264949 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.264952    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:18.265238 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.265092    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:18.265238 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.265098    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:11:18.265238 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.265127    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:11:18.265238 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.265141    2573 projected.go:196] Error preparing data for projected volume kube-api-access-6mqlc for pod openshift-network-diagnostics/network-check-target-59sxw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:18.265238 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.265155    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:34.26513635 +0000 UTC m=+33.196219652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:11:18.265238 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.265186    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc podName:1263e31c-c43e-4325-829b-e03f458e3038 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:34.265174319 +0000 UTC m=+33.196257616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mqlc" (UniqueName: "kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc") pod "network-check-target-59sxw" (UID: "1263e31c-c43e-4325-829b-e03f458e3038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:11:18.636842 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.636796    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:18.637035 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.636846    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:18.637035 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.636976    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:18.637147 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:18.637111    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:18.772309 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.772228    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 20:11:18.821607 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.821556    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pnzfv" event={"ID":"7c25adba-3b18-4022-8c5d-66c3ac0d9511","Type":"ContainerStarted","Data":"95c13d03789baefa8247bd95fec668ab07ff1fbf22da5ba8be960b6c804fbeab"}
Sep 09 20:11:18.823437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.823408    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xf98s" event={"ID":"e8e60aba-2480-4462-ac8b-15a1a89661fd","Type":"ContainerStarted","Data":"12149a4c647356b10a822454dfa7d60ef08223f2188e94537b9dc395d8a2b0d6"}
Sep 09 20:11:18.825354 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.825325    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jfvgh" event={"ID":"aeecf69a-3522-49fa-924f-3208e1e1af06","Type":"ContainerStarted","Data":"678817a715be38c2bccb79be9ceb656fe9271cf673d3c299255903d3df3cfd0d"}
Sep 09 20:11:18.827302 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.827274    2573 generic.go:356] "Generic (PLEG): container finished" podID="4f07d8e6-1c97-4b4b-96d6-027b2f9f6195" containerID="89df730b4c70be962f8dc620c6c75a3af27b99a7130368d2c8556c57cd06b411" exitCode=0
Sep 09 20:11:18.827421 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.827358    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerDied","Data":"89df730b4c70be962f8dc620c6c75a3af27b99a7130368d2c8556c57cd06b411"}
Sep 09 20:11:18.829373 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.829147    2573 generic.go:356] "Generic (PLEG): container finished" podID="842c5cd9014543c0f0425e48e33f68fb" containerID="c4df6b684556390181dda3f37f1d94aca515cae0b2fe9e2d77560ce675d63ebb" exitCode=0
Sep 09 20:11:18.829373 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.829218    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal" event={"ID":"842c5cd9014543c0f0425e48e33f68fb","Type":"ContainerDied","Data":"c4df6b684556390181dda3f37f1d94aca515cae0b2fe9e2d77560ce675d63ebb"}
Sep 09 20:11:18.831283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.831242    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd" event={"ID":"7c0456c5-dd40-4707-a5ab-97f67694fe81","Type":"ContainerStarted","Data":"e9f7a6745f0274ca4a868d8bfff3fc3cd0c971d340a3ed2cea82f28e35872fc6"}
Sep 09 20:11:18.831394 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.831311    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd" event={"ID":"7c0456c5-dd40-4707-a5ab-97f67694fe81","Type":"ContainerStarted","Data":"90642e03f84e3fcb9cdd56089309bae049cac45cbc3e190efc57ec9045c8d40a"}
Sep 09 20:11:18.837099 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:18.837052    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pnzfv" podStartSLOduration=3.020212613 podStartE2EDuration="16.83703534s" totalImagesPullingTime="13.816822727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:18.836709557 +0000 UTC m=+17.767792877" watchObservedRunningTime="2026-09-09 20:11:18.83703534 +0000 UTC m=+17.768118662"
Sep 09 20:11:19.488075 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.487967    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:11:18.77228505Z","UUID":"3407f5c6-66c6-438c-917f-7de0d894f8e3","Handler":null,"Name":"","Endpoint":""}
Sep 09 20:11:19.489855 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.489811    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 20:11:19.490465 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.490026    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 20:11:19.642272 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.638468    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:19.642272 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:19.638629    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:19.840370 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.840340    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:11:19.841368 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.841035    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"066c609fec52dd0cd13af5133e64ebbd69799be1c9bb451968880300d7a22c9e"}
Sep 09 20:11:19.843412 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.843383    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal" event={"ID":"842c5cd9014543c0f0425e48e33f68fb","Type":"ContainerStarted","Data":"c4c752843f1367f60b1add9bad4e9cf849bdc5277c63a3663b5349bd85e54269"}
Sep 09 20:11:19.861781 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:19.861725    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-68.ec2.internal" podStartSLOduration=17.861706948 podStartE2EDuration="17.861706948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:19.861616373 +0000 UTC m=+18.792699692" watchObservedRunningTime="2026-09-09 20:11:19.861706948 +0000 UTC m=+18.792790268"
Sep 09 20:11:20.386229 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.386197    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:20.636727 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.636650    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:20.636727 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.636671    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:20.636920 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:20.636773    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:20.637165 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:20.637140    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:20.849051 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.848985    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd" event={"ID":"7c0456c5-dd40-4707-a5ab-97f67694fe81","Type":"ContainerStarted","Data":"cb81560e6b3b5ca93b29cb4ffe38bc86f9f053ac16ae653f02d35d6fafb8332a"}
Sep 09 20:11:20.853616 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.853398    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:20.853995 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.853977    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:20.868662 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:20.868609    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdmqd" podStartSLOduration=3.2695616579999998 podStartE2EDuration="18.868592005s" totalImagesPullingTime="15.599030347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:20.868523525 +0000 UTC m=+19.799606845" watchObservedRunningTime="2026-09-09 20:11:20.868592005 +0000 UTC m=+19.799675325"
Sep 09 20:11:21.637073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:21.637038    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:21.637270 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:21.637152    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:21.852553 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:21.852525    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9jmtd"
Sep 09 20:11:22.636747 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:22.636488    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:22.636747 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:22.636599    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:22.636747 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:22.636630    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:22.636747 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:22.636712    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:23.636514 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.636319    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:23.636976 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:23.636616    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:23.859969 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.859947    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:11:23.860364 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.860341    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"666fcd70ed1106a4e43ea7d336e696d428c282ef2fff989b218f83666df1342a"}
Sep 09 20:11:23.860587 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.860555    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:23.860712 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.860695    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:23.860794 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.860777    2573 scope.go:117] "RemoveContainer" containerID="c4c6a0cf32e83f292993208ac035204976c2cdbd9d320d915e052f1c2d99641b"
Sep 09 20:11:23.862575 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.862555    2573 generic.go:356] "Generic (PLEG): container finished" podID="4f07d8e6-1c97-4b4b-96d6-027b2f9f6195" containerID="69be74abe29f58829bf191cfdbe025610208bd64bbc96fc7c5bf4c95dd9c4856" exitCode=0
Sep 09 20:11:23.862666 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.862590    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerDied","Data":"69be74abe29f58829bf191cfdbe025610208bd64bbc96fc7c5bf4c95dd9c4856"}
Sep 09 20:11:23.876717 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.876699    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:23.882668 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:23.882652    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:24.637508 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:24.636809    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:24.637508 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:24.636934    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:24.637508 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:24.637360    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:24.637508 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:24.637459    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:25.242056 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:25.242022    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h49zk"]
Sep 09 20:11:25.242333 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:25.242127    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:25.242333 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:25.242230    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:25.244687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:25.244663    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-59sxw"]
Sep 09 20:11:25.244828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:25.244806    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:25.244969 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:25.244899    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:25.245624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:25.245447    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5ghhl"]
Sep 09 20:11:25.245624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:25.245546    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:25.245782 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:25.245652    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:26.635828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:26.635794    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:26.635828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:26.635831    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:26.636567 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:26.635917    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:26.636567 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:26.636066    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:27.010382 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:27.010353    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:27.636671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:27.636644    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:27.636999 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:27.636743    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:27.875525 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:27.875377    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:11:27.875846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:27.875826    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" event={"ID":"98cddd01-8d4d-45e7-86d0-f8edb8d38c1a","Type":"ContainerStarted","Data":"a59b47ca92c21ad1b5689746dfdb27c5db319be67218c0145d40d8c29790dbea"}
Sep 09 20:11:27.904049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:27.903995    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb" podStartSLOduration=12.061792407 podStartE2EDuration="25.903977793s" totalImagesPullingTime="13.842185386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:27.903634051 +0000 UTC m=+26.834717551" watchObservedRunningTime="2026-09-09 20:11:27.903977793 +0000 UTC m=+26.835061112"
Sep 09 20:11:28.635836 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.635807    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:28.636003 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:28.635903    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:28.636003 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.635958    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:28.636077 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:28.636032    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:28.880629 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.880601    2573 generic.go:356] "Generic (PLEG): container finished" podID="4f07d8e6-1c97-4b4b-96d6-027b2f9f6195" containerID="bd73b99bc7d4077b39fcaaca6a8252b93a52a0b59e26162f46d60a5c6ebb334e" exitCode=0
Sep 09 20:11:28.881167 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.880641    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerDied","Data":"bd73b99bc7d4077b39fcaaca6a8252b93a52a0b59e26162f46d60a5c6ebb334e"}
Sep 09 20:11:28.882384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.882343    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7l2m7" event={"ID":"e585404d-4349-472e-ba0d-3f8bd42682d4","Type":"ContainerStarted","Data":"0a4b9f545e2848fe6351663071aea1ef05b76a46313c54d2cfb9bcd33f585e0d"}
Sep 09 20:11:28.895769 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.895749    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2frcb"
Sep 09 20:11:28.946725 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:28.946670    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7l2m7" podStartSLOduration=1.8903766960000001 podStartE2EDuration="26.946650965s" totalImagesPullingTime="25.056274269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-09 20:11:28.946173884 +0000 UTC m=+27.877257204" watchObservedRunningTime="2026-09-09 20:11:28.946650965 +0000 UTC m=+27.877734287"
Sep 09 20:11:29.636361 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:29.636333    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:29.636518 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:29.636459    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h49zk" podUID="41cf389f-73c5-4623-ba64-c2efc1782d75"
Sep 09 20:11:30.636544 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.636372    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:30.636870 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.636449    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:30.636870 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:30.636624    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-59sxw" podUID="1263e31c-c43e-4325-829b-e03f458e3038"
Sep 09 20:11:30.636870 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:30.636704    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ghhl" podUID="80a080ef-623f-4b93-be6c-2b4355b2afc2"
Sep 09 20:11:30.860524 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.860499    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-68.ec2.internal" event="NodeReady"
Sep 09 20:11:30.860650 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.860580    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 20:11:30.889431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.889367    2573 generic.go:356] "Generic (PLEG): container finished" podID="4f07d8e6-1c97-4b4b-96d6-027b2f9f6195" containerID="6efd69f1aa3a76171a58a157b489b00f955b96136abbe56a0171bfcfdc703ccd" exitCode=0
Sep 09 20:11:30.889431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.889406    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerDied","Data":"6efd69f1aa3a76171a58a157b489b00f955b96136abbe56a0171bfcfdc703ccd"}
Sep 09 20:11:30.899398 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.899380    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-755fdc4cbc-ndxnf"]
Sep 09 20:11:30.941097 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.941073    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dkb4d"]
Sep 09 20:11:30.941203 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.941187    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:30.944856 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.944838    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 20:11:30.944940 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.944921    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gslg8\""
Sep 09 20:11:30.945010 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.944987    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 20:11:30.945322 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.945306    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 20:11:30.952026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.952002    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 20:11:30.964936 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.964919    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rr9dx"]
Sep 09 20:11:30.965025 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.965012    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:30.967979 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.967911    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ftsch\""
Sep 09 20:11:30.968068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.968007    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 20:11:30.968381 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.968283    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 20:11:30.988733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.988717    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-755fdc4cbc-ndxnf"]
Sep 09 20:11:30.988817 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.988742    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkb4d"]
Sep 09 20:11:30.988817 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.988759    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rr9dx"]
Sep 09 20:11:30.988885 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.988827    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:30.991428 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.991408    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 20:11:30.991518 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.991413    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 20:11:30.991518 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.991440    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 20:11:30.991518 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:30.991489    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7rf7m\""
Sep 09 20:11:31.060873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.060853    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c72e7028-68f7-4111-8461-15906e90decc-ca-trust-extracted\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.060954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.060878    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz55q\" (UniqueName: \"kubernetes.io/projected/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-kube-api-access-gz55q\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.060954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.060896    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e682d8e2-2fc0-468d-8db5-1e1f6372a43c-cert\") pod \"ingress-canary-rr9dx\" (UID: \"e682d8e2-2fc0-468d-8db5-1e1f6372a43c\") " pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.060954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.060918    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-image-registry-private-configuration\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.061058 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061002    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hhb5\" (UniqueName: \"kubernetes.io/projected/e682d8e2-2fc0-468d-8db5-1e1f6372a43c-kube-api-access-2hhb5\") pod \"ingress-canary-rr9dx\" (UID: \"e682d8e2-2fc0-468d-8db5-1e1f6372a43c\") " pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.061058 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061023    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-trusted-ca\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.061058 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061043    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqltz\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-kube-api-access-pqltz\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.061143 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061059    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-config-volume\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.061143 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061115    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.061204 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061144    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-tmp-dir\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.061204 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061172    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-bound-sa-token\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.061281 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061205    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.061281 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061225    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-registry-certificates\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.061281 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.061263    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-installation-pull-secrets\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.104769 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.104739    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:31.161890 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.161847    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c72e7028-68f7-4111-8461-15906e90decc-ca-trust-extracted\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.161890 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.161873    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz55q\" (UniqueName: \"kubernetes.io/projected/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-kube-api-access-gz55q\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.161981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.161892    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e682d8e2-2fc0-468d-8db5-1e1f6372a43c-cert\") pod \"ingress-canary-rr9dx\" (UID: \"e682d8e2-2fc0-468d-8db5-1e1f6372a43c\") " pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.161981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.161912    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-image-registry-private-configuration\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.161979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hhb5\" (UniqueName: \"kubernetes.io/projected/e682d8e2-2fc0-468d-8db5-1e1f6372a43c-kube-api-access-2hhb5\") pod \"ingress-canary-rr9dx\" (UID: \"e682d8e2-2fc0-468d-8db5-1e1f6372a43c\") " pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.162049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.161998    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-trusted-ca\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162219 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162199    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqltz\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-kube-api-access-pqltz\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162299 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162240    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-config-volume\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.162405 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162388    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c72e7028-68f7-4111-8461-15906e90decc-ca-trust-extracted\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162405 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162393    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.162508 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162427    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-tmp-dir\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.162508 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162460    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-bound-sa-token\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162508 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.162485    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:11:31.162508 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162499    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162698 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162531    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-registry-certificates\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162698 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.162555    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:31.662530849 +0000 UTC m=+30.593614154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:11:31.162698 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162598    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-installation-pull-secrets\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.162848 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162775    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-tmp-dir\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.162848 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.162803    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-config-volume\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.163037 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.163019    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:11:31.163106 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.163039    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:11:31.163106 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.163083    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-registry-certificates\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.163106 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.163100    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:11:31.663084861 +0000 UTC m=+30.594168186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:11:31.163217 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.163162    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-trusted-ca\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.166158 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.166138    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-installation-pull-secrets\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.166297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.166278    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-image-registry-private-configuration\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.166337 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.166323    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e682d8e2-2fc0-468d-8db5-1e1f6372a43c-cert\") pod \"ingress-canary-rr9dx\" (UID: \"e682d8e2-2fc0-468d-8db5-1e1f6372a43c\") " pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.172100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.172077    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqltz\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-kube-api-access-pqltz\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.172302 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.172282    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-bound-sa-token\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.172728 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.172707    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz55q\" (UniqueName: \"kubernetes.io/projected/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-kube-api-access-gz55q\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.172924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.172908    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hhb5\" (UniqueName: \"kubernetes.io/projected/e682d8e2-2fc0-468d-8db5-1e1f6372a43c-kube-api-access-2hhb5\") pod \"ingress-canary-rr9dx\" (UID: \"e682d8e2-2fc0-468d-8db5-1e1f6372a43c\") " pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.197087 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.197063    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:31.296160 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.296128    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rr9dx"
Sep 09 20:11:31.615746 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.615714    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rr9dx"]
Sep 09 20:11:31.619447 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:31.619419    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode682d8e2_2fc0_468d_8db5_1e1f6372a43c.slice/crio-8106fe4f6cc0beb512eebf38cd809b01f0ece531976a974af4e556d3a1042688 WatchSource:0}: Error finding container 8106fe4f6cc0beb512eebf38cd809b01f0ece531976a974af4e556d3a1042688: Status 404 returned error can't find the container with id 8106fe4f6cc0beb512eebf38cd809b01f0ece531976a974af4e556d3a1042688
Sep 09 20:11:31.637003 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.636980    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:31.639563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.639541    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:11:31.666705 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.666678    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:31.666809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.666713    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:31.666853 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.666818    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:11:31.666853 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.666820    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:11:31.666928 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.666891    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:32.66687137 +0000 UTC m=+31.597954680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:11:31.666928 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.666829    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:11:31.667025 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:31.666964    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:11:32.666949054 +0000 UTC m=+31.598032351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:11:31.893458 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:31.893377    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rr9dx" event={"ID":"e682d8e2-2fc0-468d-8db5-1e1f6372a43c","Type":"ContainerStarted","Data":"8106fe4f6cc0beb512eebf38cd809b01f0ece531976a974af4e556d3a1042688"}
Sep 09 20:11:32.636687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.636650    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:32.636830 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.636702    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:32.639609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.639585    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:11:32.641174 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.640836    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4t9vt\""
Sep 09 20:11:32.641174 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.640902    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:11:32.641174 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.641064    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:11:32.641174 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.641099    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wr6wr\""
Sep 09 20:11:32.675500 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.675469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:32.675621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.675527    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:32.675621 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:32.675593    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:11:32.675699 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:32.675674    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:34.675651042 +0000 UTC m=+33.606734354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:11:32.675699 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:32.675684    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:11:32.675790 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:32.675702    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:11:32.675790 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:32.675764    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:11:34.675745526 +0000 UTC m=+33.606828842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:11:32.925756 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:32.925430    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:33.010704 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:33.010667    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:34.186591 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.186510    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:34.189533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.189508    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41cf389f-73c5-4623-ba64-c2efc1782d75-original-pull-secret\") pod \"global-pull-secret-syncer-h49zk\" (UID: \"41cf389f-73c5-4623-ba64-c2efc1782d75\") " pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:34.287732 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.287691    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:34.287732 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.287738    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:11:34.287961 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.287865    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 20:11:34.287961 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.287932    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:06.287913302 +0000 UTC m=+65.218996622 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : secret "metrics-daemon-secret" not found
Sep 09 20:11:34.290860 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.290811    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mqlc\" (UniqueName: \"kubernetes.io/projected/1263e31c-c43e-4325-829b-e03f458e3038-kube-api-access-6mqlc\") pod \"network-check-target-59sxw\" (UID: \"1263e31c-c43e-4325-829b-e03f458e3038\") " pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:34.347769 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.347730    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h49zk"
Sep 09 20:11:34.456880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.456795    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:34.690356 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.690323    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:34.690533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:34.690372    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:34.690533 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.690478    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:11:34.690645 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.690556    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:38.690536085 +0000 UTC m=+37.621619400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:11:34.690645 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.690480    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:11:34.690645 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.690585    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:11:34.690645 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:34.690637    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:11:38.690622459 +0000 UTC m=+37.621705756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:11:35.179267 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:35.179210    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:35.238192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:35.238161    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:35.567436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:35.567351    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:35.667732 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:35.667701    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:36.056419 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.056388    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:36.125904 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.125865    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:36.398602 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.398524    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h49zk"]
Sep 09 20:11:36.405666 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.404593    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-59sxw"]
Sep 09 20:11:36.442426 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:36.442392    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1263e31c_c43e_4325_829b_e03f458e3038.slice/crio-780830fa8762152e8130a9596db510e004b404cd395ef3480b587e6d9f7fb2c1 WatchSource:0}: Error finding container 780830fa8762152e8130a9596db510e004b404cd395ef3480b587e6d9f7fb2c1: Status 404 returned error can't find the container with id 780830fa8762152e8130a9596db510e004b404cd395ef3480b587e6d9f7fb2c1
Sep 09 20:11:36.446916 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:11:36.446895    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41cf389f_73c5_4623_ba64_c2efc1782d75.slice/crio-6ba87cc02e072e313b07b9a5a90b003980c81ba9313ad883cdef99b74d9b7166 WatchSource:0}: Error finding container 6ba87cc02e072e313b07b9a5a90b003980c81ba9313ad883cdef99b74d9b7166: Status 404 returned error can't find the container with id 6ba87cc02e072e313b07b9a5a90b003980c81ba9313ad883cdef99b74d9b7166
Sep 09 20:11:36.846562 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.846531    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:36.906604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.906570    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-59sxw" event={"ID":"1263e31c-c43e-4325-829b-e03f458e3038","Type":"ContainerStarted","Data":"780830fa8762152e8130a9596db510e004b404cd395ef3480b587e6d9f7fb2c1"}
Sep 09 20:11:36.907752 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.907729    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h49zk" event={"ID":"41cf389f-73c5-4623-ba64-c2efc1782d75","Type":"ContainerStarted","Data":"6ba87cc02e072e313b07b9a5a90b003980c81ba9313ad883cdef99b74d9b7166"}
Sep 09 20:11:36.909145 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.909122    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rr9dx" event={"ID":"e682d8e2-2fc0-468d-8db5-1e1f6372a43c","Type":"ContainerStarted","Data":"e20ce529184d099d85ee817f0c4e0c9368c6d1fde1abf09ebaaf2efed5217b6c"}
Sep 09 20:11:36.911705 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.911684    2573 generic.go:356] "Generic (PLEG): container finished" podID="4f07d8e6-1c97-4b4b-96d6-027b2f9f6195" containerID="9b4331faae5c6e0d48a113bf9acafce8e8e7ee6d7a41a01edd9890d2dc522b0c" exitCode=0
Sep 09 20:11:36.911792 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.911714    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerDied","Data":"9b4331faae5c6e0d48a113bf9acafce8e8e7ee6d7a41a01edd9890d2dc522b0c"}
Sep 09 20:11:36.925340 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.925303    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rr9dx" podStartSLOduration=2.415924673 podStartE2EDuration="6.925289033s" totalImagesPullingTime="4.50936436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:36.924094288 +0000 UTC m=+35.855177608" watchObservedRunningTime="2026-09-09 20:11:36.925289033 +0000 UTC m=+35.856372351"
Sep 09 20:11:36.932516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:36.932496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:37.759730 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:37.759686    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:37.881776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:37.881730    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:37.919695 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:37.919663    2573 generic.go:356] "Generic (PLEG): container finished" podID="4f07d8e6-1c97-4b4b-96d6-027b2f9f6195" containerID="4656c965ce1687eb3dd18506c6fac5b368b8ac83dcbe80458843a5be92efb57c" exitCode=0
Sep 09 20:11:37.919849 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:37.919739    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerDied","Data":"4656c965ce1687eb3dd18506c6fac5b368b8ac83dcbe80458843a5be92efb57c"}
Sep 09 20:11:37.945636 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:37.945605    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:38.723272 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:38.723213    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:38.723272 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:38.723272    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:38.723464 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:38.723358    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:11:38.723464 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:38.723372    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:11:38.723464 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:38.723415    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:11:46.723402814 +0000 UTC m=+45.654486111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:11:38.723464 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:38.723358    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:11:38.723613 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:38.723490    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:11:46.723477566 +0000 UTC m=+45.654560864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:11:38.796495 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:38.796458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:38.859798 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:38.859761    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:38.927154 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:38.927122    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" event={"ID":"4f07d8e6-1c97-4b4b-96d6-027b2f9f6195","Type":"ContainerStarted","Data":"9c8744bb2e3caf8d7c9d082bb5ad2e428d70598c2d2cd6f31dc31eeb3c76b9d1"}
Sep 09 20:11:38.950788 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:38.950730    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vhb7f" podStartSLOduration=7.932212844 podStartE2EDuration="36.950713212s" totalImagesPullingTime="29.018500368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:38.94951233 +0000 UTC m=+37.880595671" watchObservedRunningTime="2026-09-09 20:11:38.950713212 +0000 UTC m=+37.881796531"
Sep 09 20:11:39.082054 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:39.081944    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:39.180353 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:39.180310    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:39.245820 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:39.245786    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:40.753872 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:40.753815    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:41.936646 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:41.936621    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h49zk" event={"ID":"41cf389f-73c5-4623-ba64-c2efc1782d75","Type":"ContainerStarted","Data":"6e71429b1bf105760f3361ec39ce7f7a2e89a11843905df5ab9472d5de7c4ba8"}
Sep 09 20:11:41.938284 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:41.938238    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-59sxw" event={"ID":"1263e31c-c43e-4325-829b-e03f458e3038","Type":"ContainerStarted","Data":"8757b63ab87591589e77a691c6517e66247651e4946b24de7a9705240d0d4ad3"}
Sep 09 20:11:41.938458 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:41.938435    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:11:41.958298 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:41.958221    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-59sxw" podStartSLOduration=34.626898824 podStartE2EDuration="39.958208437s" totalImagesPullingTime="5.331309613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:41.957505258 +0000 UTC m=+40.888588576" watchObservedRunningTime="2026-09-09 20:11:41.958208437 +0000 UTC m=+40.889291755"
Sep 09 20:11:42.958910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:42.958861    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h49zk" podStartSLOduration=35.62038896 podStartE2EDuration="40.958844091s" totalImagesPullingTime="5.338455131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:11:42.958531746 +0000 UTC m=+41.889615065" watchObservedRunningTime="2026-09-09 20:11:42.958844091 +0000 UTC m=+41.889927400"
Sep 09 20:11:46.776398 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:46.776364    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:11:46.776398 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:46.776399    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:11:46.776911 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:46.776484    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:11:46.776911 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:46.776494    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:11:46.776911 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:46.776524    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:11:46.776911 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:46.776549    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:12:02.77653646 +0000 UTC m=+61.707619758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:11:46.776911 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:11:46.776585    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:02.776567408 +0000 UTC m=+61.707650719 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:11:50.949050 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:50.949017    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:51.007998 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:51.007972    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:53.873923 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:53.873873    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:53.976200 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:53.976178    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:54.513177 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:54.513138    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:54.574169 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:54.574141    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:58.040490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:58.040454    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:11:58.108187 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:11:58.108158    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:01.731006 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.730974    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-69ggj"]
Sep 09 20:12:01.756139 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.756110    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"]
Sep 09 20:12:01.756297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.756222    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:01.759841 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.759820    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hqnmq\""
Sep 09 20:12:01.759976 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.759904    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 20:12:01.760541 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.760526    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 20:12:01.770236 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.770215    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-69ggj"]
Sep 09 20:12:01.770347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.770259    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"]
Sep 09 20:12:01.770347 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.770336    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.772961 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.772945    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 20:12:01.773057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.772968    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zt7ph\""
Sep 09 20:12:01.773946 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.773929    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 09 20:12:01.774036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.773998    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 20:12:01.774094 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.774071    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 09 20:12:01.862428 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.862388    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dm4lq"]
Sep 09 20:12:01.874971 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.874939    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:01.878040 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.878014    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.878147 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.878047    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:01.878147 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.878066    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.878147 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.878102    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-nginx-conf\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:01.878147 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.878122    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nngqs\" (UniqueName: \"kubernetes.io/projected/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-kube-api-access-nngqs\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.885530 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.885507    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.885624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.885577    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dm4lq"]
Sep 09 20:12:01.887959 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.887939    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 20:12:01.888060 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.887966    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 09 20:12:01.888299 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.888278    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 09 20:12:01.888399 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.888379    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lm6r4\""
Sep 09 20:12:01.888459 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.888390    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 20:12:01.892713 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.892692    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 09 20:12:01.979422 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979383    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-snapshots\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.979624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979438    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.979624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:01.979624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979541    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.979624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979575    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-tmp\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.979624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979614    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb625\" (UniqueName: \"kubernetes.io/projected/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-kube-api-access-bb625\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-nginx-conf\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nngqs\" (UniqueName: \"kubernetes.io/projected/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-kube-api-access-nngqs\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:01.979682    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:01.979697    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979696    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-serving-cert\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:01.979763    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert podName:21f2a60f-d391-409c-a92f-a4b3ea9b9c9b nodeName:}" failed. No retries permitted until 2026-09-09 20:12:02.479740588 +0000 UTC m=+61.410823888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-69ggj" (UID: "21f2a60f-d391-409c-a92f-a4b3ea9b9c9b") : secret "networking-console-plugin-cert" not found
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:01.979810    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls podName:7d96ce2f-0d16-4c53-a1f0-9b4e5d363441 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:02.479787068 +0000 UTC m=+61.410870367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9p225" (UID: "7d96ce2f-0d16-4c53-a1f0-9b4e5d363441") : secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:01.979880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979859    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.980210 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.979904    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:01.980210 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.980109    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:01.980302 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.980224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-nginx-conf\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:01.990017 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:01.989957    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nngqs\" (UniqueName: \"kubernetes.io/projected/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-kube-api-access-nngqs\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:02.080708 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.080664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-serving-cert\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.080708 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.080716    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.080947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.080789    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.080947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.080850    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-snapshots\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.080947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.080897    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-tmp\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.080947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.080919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb625\" (UniqueName: \"kubernetes.io/projected/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-kube-api-access-bb625\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.081527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.081505    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.081630 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.081505    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-tmp\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.081630 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.081615    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-snapshots\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.081801 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.081783    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.083234 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.083212    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-serving-cert\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.089878 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.089857    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb625\" (UniqueName: \"kubernetes.io/projected/0dd432c4-c58f-4523-a3a0-6ca0b6e27366-kube-api-access-bb625\") pod \"insights-operator-76bcd6ccf5-dm4lq\" (UID: \"0dd432c4-c58f-4523-a3a0-6ca0b6e27366\") " pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.195818 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.195785    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq"
Sep 09 20:12:02.234217 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.234188    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:02.320775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.320741    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:02.484827 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.484794    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:02.484827 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.484832    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:02.485025 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.484956    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:12:02.485025 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.485013    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert podName:21f2a60f-d391-409c-a92f-a4b3ea9b9c9b nodeName:}" failed. No retries permitted until 2026-09-09 20:12:03.484999749 +0000 UTC m=+62.416083051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-69ggj" (UID: "21f2a60f-d391-409c-a92f-a4b3ea9b9c9b") : secret "networking-console-plugin-cert" not found
Sep 09 20:12:02.485110 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.484961    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:02.485110 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.485094    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls podName:7d96ce2f-0d16-4c53-a1f0-9b4e5d363441 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:03.485080793 +0000 UTC m=+62.416164099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9p225" (UID: "7d96ce2f-0d16-4c53-a1f0-9b4e5d363441") : secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:02.491024 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.490992    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-dm4lq"]
Sep 09 20:12:02.493363 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:02.493341    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd432c4_c58f_4523_a3a0_6ca0b6e27366.slice/crio-9524c6a8e84d0fc9efd887227b8e482c3177223c50d851626396867ecc025a09 WatchSource:0}: Error finding container 9524c6a8e84d0fc9efd887227b8e482c3177223c50d851626396867ecc025a09: Status 404 returned error can't find the container with id 9524c6a8e84d0fc9efd887227b8e482c3177223c50d851626396867ecc025a09
Sep 09 20:12:02.787564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.787530    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:12:02.787986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.787585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:12:02.787986 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.787682    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:12:02.787986 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.787715    2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:12:02.787986 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.787728    2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755fdc4cbc-ndxnf: secret "image-registry-tls" not found
Sep 09 20:12:02.787986 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.787746    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls podName:bdee7bc4-c3c5-4ee2-9913-f967f8649d42 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:34.787731159 +0000 UTC m=+93.718814455 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls") pod "dns-default-dkb4d" (UID: "bdee7bc4-c3c5-4ee2-9913-f967f8649d42") : secret "dns-default-metrics-tls" not found
Sep 09 20:12:02.787986 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:02.787770    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls podName:c72e7028-68f7-4111-8461-15906e90decc nodeName:}" failed. No retries permitted until 2026-09-09 20:12:34.787757709 +0000 UTC m=+93.718841011 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls") pod "image-registry-755fdc4cbc-ndxnf" (UID: "c72e7028-68f7-4111-8461-15906e90decc") : secret "image-registry-tls" not found
Sep 09 20:12:02.989073 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:02.989033    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq" event={"ID":"0dd432c4-c58f-4523-a3a0-6ca0b6e27366","Type":"ContainerStarted","Data":"9524c6a8e84d0fc9efd887227b8e482c3177223c50d851626396867ecc025a09"}
Sep 09 20:12:03.492974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:03.492940    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:03.493168 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:03.492981    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:03.493168 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:03.493102    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:03.493168 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:03.493105    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:12:03.493168 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:03.493160    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls podName:7d96ce2f-0d16-4c53-a1f0-9b4e5d363441 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:05.493145229 +0000 UTC m=+64.424228526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9p225" (UID: "7d96ce2f-0d16-4c53-a1f0-9b4e5d363441") : secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:03.493374 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:03.493178    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert podName:21f2a60f-d391-409c-a92f-a4b3ea9b9c9b nodeName:}" failed. No retries permitted until 2026-09-09 20:12:05.493169551 +0000 UTC m=+64.424252852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-69ggj" (UID: "21f2a60f-d391-409c-a92f-a4b3ea9b9c9b") : secret "networking-console-plugin-cert" not found
Sep 09 20:12:04.108646 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:04.108605    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:04.170726 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:04.170699    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:04.995680 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:04.995645    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq" event={"ID":"0dd432c4-c58f-4523-a3a0-6ca0b6e27366","Type":"ContainerStarted","Data":"69aee83df5048b49edbb9d6e062eb3a967c753b74b18ace3b85e4f67ce5ec3df"}
Sep 09 20:12:05.013016 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.012911    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq" podStartSLOduration=1.749753278 podStartE2EDuration="4.012896447s" totalImagesPullingTime="2.263143169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:05.011875539 +0000 UTC m=+63.942958858" watchObservedRunningTime="2026-09-09 20:12:05.012896447 +0000 UTC m=+63.943979766"
Sep 09 20:12:05.511814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.511760    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:05.511814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.511812    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:05.512372 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.511928    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:12:05.512372 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.511945    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:05.512372 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.511996    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls podName:7d96ce2f-0d16-4c53-a1f0-9b4e5d363441 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:09.511982382 +0000 UTC m=+68.443065678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9p225" (UID: "7d96ce2f-0d16-4c53-a1f0-9b4e5d363441") : secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:05.512372 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.512011    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert podName:21f2a60f-d391-409c-a92f-a4b3ea9b9c9b nodeName:}" failed. No retries permitted until 2026-09-09 20:12:09.512005098 +0000 UTC m=+68.443088394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-69ggj" (UID: "21f2a60f-d391-409c-a92f-a4b3ea9b9c9b") : secret "networking-console-plugin-cert" not found
Sep 09 20:12:05.685236 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.685202    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gtv5p"]
Sep 09 20:12:05.687212 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.687196    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.689514 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.689491    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 20:12:05.689659 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.689517    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 20:12:05.689659 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.689523    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j97b5\""
Sep 09 20:12:05.698360 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.698332    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gtv5p"]
Sep 09 20:12:05.707629 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.707593    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:05.763662 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.763587    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:05.814810 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.814780    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.814947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.814829    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d97dp\" (UniqueName: \"kubernetes.io/projected/0de91534-394e-41b0-b58f-a199613bc5f6-kube-api-access-d97dp\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.814947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.814896    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0de91534-394e-41b0-b58f-a199613bc5f6-data-volume\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.814947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.814918    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0de91534-394e-41b0-b58f-a199613bc5f6-crio-socket\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.814947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.814941    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.915944 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.915913    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0de91534-394e-41b0-b58f-a199613bc5f6-data-volume\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.915944 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.915945    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0de91534-394e-41b0-b58f-a199613bc5f6-crio-socket\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.916120 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.915972    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.916120 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.916031    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.916120 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.916061    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0de91534-394e-41b0-b58f-a199613bc5f6-crio-socket\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.916215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.916119    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d97dp\" (UniqueName: \"kubernetes.io/projected/0de91534-394e-41b0-b58f-a199613bc5f6-kube-api-access-d97dp\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.916285 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.916267    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:05.916319 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.916302    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:12:05.916351 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.916332    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:06.416316611 +0000 UTC m=+65.347399913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:05.916393 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:05.916352    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:06.416339867 +0000 UTC m=+65.347423167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : configmap "kube-rbac-proxy" not found
Sep 09 20:12:05.916393 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.916370    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0de91534-394e-41b0-b58f-a199613bc5f6-data-volume\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:05.924648 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:05.924624    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d97dp\" (UniqueName: \"kubernetes.io/projected/0de91534-394e-41b0-b58f-a199613bc5f6-kube-api-access-d97dp\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:06.318825 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.318788    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:12:06.319005 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:06.318903    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 20:12:06.319005 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:06.318955    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs podName:80a080ef-623f-4b93-be6c-2b4355b2afc2 nodeName:}" failed. No retries permitted until 2026-09-09 20:13:10.318940511 +0000 UTC m=+129.250023809 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs") pod "network-metrics-daemon-5ghhl" (UID: "80a080ef-623f-4b93-be6c-2b4355b2afc2") : secret "metrics-daemon-secret" not found
Sep 09 20:12:06.419383 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.419348    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:06.419554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.419408    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:06.419554 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:06.419467    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:12:06.419554 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:06.419532    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:07.419517527 +0000 UTC m=+66.350600824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : configmap "kube-rbac-proxy" not found
Sep 09 20:12:06.419554 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:06.419539    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:06.419712 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:06.419576    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:07.419566166 +0000 UTC m=+66.350649463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:06.725215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.725184    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:06.788450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.788417    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:06.939799 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.939766    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:06.999437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:06.999353    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:07.421809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.421766    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:07.427903 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.427881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:07.428002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.427987    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:07.428044 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:07.428015    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:07.428088 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:07.428065    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:09.428051651 +0000 UTC m=+68.359134965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:07.428088 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:07.428068    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:12:07.428175 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:07.428112    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:09.428096785 +0000 UTC m=+68.359180105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : configmap "kube-rbac-proxy" not found
Sep 09 20:12:07.446179 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.446153    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jfvgh_aeecf69a-3522-49fa-924f-3208e1e1af06/dns-node-resolver/0.log"
Sep 09 20:12:07.521227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.521192    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:07.847942 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.847843    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:07.949700 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:07.949667    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:08.773427 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:08.773392    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:08.843354 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:08.843324    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:08.847950 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:08.847933    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xf98s_e8e60aba-2480-4462-ac8b-15a1a89661fd/node-ca/0.log"
Sep 09 20:12:09.016563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.016537    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-fg7mp"]
Sep 09 20:12:09.048534 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.048478    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-fg7mp"]
Sep 09 20:12:09.048666 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.048566    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.051557 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.051537    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 09 20:12:09.051758 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.051739    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 09 20:12:09.052725 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.052704    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 09 20:12:09.052814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.052778    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2q5bd\""
Sep 09 20:12:09.052814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.052782    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 09 20:12:09.120299 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.120268    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:09.138292 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.138245    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/22ab8840-f889-4172-9441-b609b5d80230-signing-key\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.138388 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.138339    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl9d9\" (UniqueName: \"kubernetes.io/projected/22ab8840-f889-4172-9441-b609b5d80230-kube-api-access-rl9d9\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.138444 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.138419    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/22ab8840-f889-4172-9441-b609b5d80230-signing-cabundle\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.222117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.222089    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:09.239629 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.239604    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/22ab8840-f889-4172-9441-b609b5d80230-signing-key\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.239714 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.239644    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl9d9\" (UniqueName: \"kubernetes.io/projected/22ab8840-f889-4172-9441-b609b5d80230-kube-api-access-rl9d9\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.239714 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.239699    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/22ab8840-f889-4172-9441-b609b5d80230-signing-cabundle\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.240428 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.240410    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/22ab8840-f889-4172-9441-b609b5d80230-signing-cabundle\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.241961 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.241942    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/22ab8840-f889-4172-9441-b609b5d80230-signing-key\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.248286 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.248267    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl9d9\" (UniqueName: \"kubernetes.io/projected/22ab8840-f889-4172-9441-b609b5d80230-kube-api-access-rl9d9\") pod \"service-ca-75d99c484-fg7mp\" (UID: \"22ab8840-f889-4172-9441-b609b5d80230\") " pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.357450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.357429    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-fg7mp"
Sep 09 20:12:09.441375 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.441225    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:09.442184 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.441348    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:12:09.442184 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.441814    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:13.441789186 +0000 UTC m=+72.372872502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : configmap "kube-rbac-proxy" not found
Sep 09 20:12:09.442184 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.442037    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:09.442400 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.442198    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:09.442400 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.442287    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:13.442269308 +0000 UTC m=+72.373352622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:09.543082 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.543059    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:09.543241 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.543089    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:09.543241 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.543204    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:09.543241 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.543212    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:12:09.543398 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.543273    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls podName:7d96ce2f-0d16-4c53-a1f0-9b4e5d363441 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:17.54323556 +0000 UTC m=+76.474318875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9p225" (UID: "7d96ce2f-0d16-4c53-a1f0-9b4e5d363441") : secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:09.543398 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:09.543296    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert podName:21f2a60f-d391-409c-a92f-a4b3ea9b9c9b nodeName:}" failed. No retries permitted until 2026-09-09 20:12:17.543279274 +0000 UTC m=+76.474362588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-69ggj" (UID: "21f2a60f-d391-409c-a92f-a4b3ea9b9c9b") : secret "networking-console-plugin-cert" not found
Sep 09 20:12:09.646550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.646497    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-fg7mp"]
Sep 09 20:12:09.648111 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.648096    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rr9dx_e682d8e2-2fc0-468d-8db5-1e1f6372a43c/serve-healthcheck-canary/0.log"
Sep 09 20:12:09.651024 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:09.651005    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ab8840_f889_4172_9441_b609b5d80230.slice/crio-5d2d6ab2cdd474ae0046561d201d8700e11f8cd5698c53515da86d2a9561e95b WatchSource:0}: Error finding container 5d2d6ab2cdd474ae0046561d201d8700e11f8cd5698c53515da86d2a9561e95b: Status 404 returned error can't find the container with id 5d2d6ab2cdd474ae0046561d201d8700e11f8cd5698c53515da86d2a9561e95b
Sep 09 20:12:09.980983 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:09.980912    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:10.007787 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:10.007760    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-fg7mp" event={"ID":"22ab8840-f889-4172-9441-b609b5d80230","Type":"ContainerStarted","Data":"5d2d6ab2cdd474ae0046561d201d8700e11f8cd5698c53515da86d2a9561e95b"}
Sep 09 20:12:12.908116 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:12.908078    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:12.944075 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:12.943907    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-59sxw"
Sep 09 20:12:13.024935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.024902    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-fg7mp" event={"ID":"22ab8840-f889-4172-9441-b609b5d80230","Type":"ContainerStarted","Data":"52011b228622a647d579ef7921c14a43eeea78c8088b443182b3acdde52ad752"}
Sep 09 20:12:13.032508 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.032478    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:13.045543 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.045503    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-fg7mp" podStartSLOduration=1.6199149560000001 podStartE2EDuration="4.045489289s" totalImagesPullingTime="2.425574333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:13.044513545 +0000 UTC m=+71.975596862" watchObservedRunningTime="2026-09-09 20:12:13.045489289 +0000 UTC m=+71.976572660"
Sep 09 20:12:13.099994 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.099973    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:13.238839 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.238779    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:13.307945 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.307916    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:13.475468 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.475435    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:13.475671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.475504    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:13.475671 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:13.475573    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:12:13.475671 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:13.475616    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:13.475671 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:13.475649    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:21.475628158 +0000 UTC m=+80.406711458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : configmap "kube-rbac-proxy" not found
Sep 09 20:12:13.475864 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:13.475678    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:21.475661099 +0000 UTC m=+80.406744411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:13.600982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:13.600917    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:14.078490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:14.078447    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:14.394847 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:14.394806    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:14.788289 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:14.788206    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:15.198611 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:15.198582    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:15.698466 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:15.698437    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:15.800548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:15.800520    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:16.387554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:16.387493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:16.581189 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:16.581161    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:16.761711 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:16.761642    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:16.816840 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:16.816812    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:17.219053 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:17.219023    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:17.399318 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:17.399284    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:17.500341 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:17.500276    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:17.600676 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:17.600646    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:17.607664 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:17.607639    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:17.607793 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:17.607668    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:17.607851 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:17.607807    2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:12:17.607851 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:17.607816    2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:17.607957 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:17.607871    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls podName:7d96ce2f-0d16-4c53-a1f0-9b4e5d363441 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:33.607853516 +0000 UTC m=+92.538936829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9p225" (UID: "7d96ce2f-0d16-4c53-a1f0-9b4e5d363441") : secret "cluster-monitoring-operator-tls" not found
Sep 09 20:12:17.607957 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:17.607890    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert podName:21f2a60f-d391-409c-a92f-a4b3ea9b9c9b nodeName:}" failed. No retries permitted until 2026-09-09 20:12:33.607880282 +0000 UTC m=+92.538963579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-69ggj" (UID: "21f2a60f-d391-409c-a92f-a4b3ea9b9c9b") : secret "networking-console-plugin-cert" not found
Sep 09 20:12:18.591476 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:18.591434    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:18.840095 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:18.840070    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:18.986100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:18.986066    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:19.193618 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:19.193585    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:19.286434 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:19.286365    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:19.390107 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:19.390076    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:19.599069 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:19.598985    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:20.387161 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:20.387130    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:20.566178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:20.566151    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:20.628214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:20.628185    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:20.987206 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:20.987178    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:21.537826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:21.537793    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:21.537993 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:21.537921    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:21.537993 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:21.537950    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:21.538067 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:21.538013    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls podName:0de91534-394e-41b0-b58f-a199613bc5f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:37.53799628 +0000 UTC m=+96.469079590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gtv5p" (UID: "0de91534-394e-41b0-b58f-a199613bc5f6") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:12:21.538514 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:21.538496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0de91534-394e-41b0-b58f-a199613bc5f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:21.659906 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:21.659875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:21.891803 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:21.891775    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:21.991811 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:21.991782    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:22.387150 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:22.387115    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:22.581857 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:22.581827    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:22.992581 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:22.992539    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:23.387091 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:23.387038    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:23.629431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:23.629396    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:29.404912 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.404882    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"]
Sep 09 20:12:29.407080 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.407061    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.411237 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.411211    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 20:12:29.411391 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.411211    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 20:12:29.411391 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.411311    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 20:12:29.411391 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.411350    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 20:12:29.423379 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.423353    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"]
Sep 09 20:12:29.493605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.493574    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:29.497613 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.497590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgdbr\" (UniqueName: \"kubernetes.io/projected/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-kube-api-access-qgdbr\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.497715 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.497704    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-tmp\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.497757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.497731    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-klusterlet-config\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.595953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.595922    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:29.598992 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.598974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-tmp\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.599057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.599009    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-klusterlet-config\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.599057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.599043    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgdbr\" (UniqueName: \"kubernetes.io/projected/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-kube-api-access-qgdbr\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.599445 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.599425    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-tmp\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.601613 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.601589    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-klusterlet-config\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.608184 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.608158    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgdbr\" (UniqueName: \"kubernetes.io/projected/0d06bb0a-4fb3-4c51-b6b9-5c0120b82233-kube-api-access-qgdbr\") pod \"klusterlet-addon-workmgr-75c77f4f4b-cz268\" (UID: \"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.693851 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.693754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:29.716048 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.716022    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:29.853429 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.853396    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:29.982941 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:29.982698    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"]
Sep 09 20:12:29.985395 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:29.985356    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d06bb0a_4fb3_4c51_b6b9_5c0120b82233.slice/crio-f18bce4b5c0a4ddc97fd41a0fe624b48d3e818d1bc7f33de650323787cf6d122 WatchSource:0}: Error finding container f18bce4b5c0a4ddc97fd41a0fe624b48d3e818d1bc7f33de650323787cf6d122: Status 404 returned error can't find the container with id f18bce4b5c0a4ddc97fd41a0fe624b48d3e818d1bc7f33de650323787cf6d122
Sep 09 20:12:30.080187 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:30.080140    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268" event={"ID":"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233","Type":"ContainerStarted","Data":"f18bce4b5c0a4ddc97fd41a0fe624b48d3e818d1bc7f33de650323787cf6d122"}
Sep 09 20:12:33.633018 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:33.632986    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:33.633018 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:33.633023    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:33.635219 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:33.635198    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21f2a60f-d391-409c-a92f-a4b3ea9b9c9b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-69ggj\" (UID: \"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:33.635370 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:33.635351    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7d96ce2f-0d16-4c53-a1f0-9b4e5d363441-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9p225\" (UID: \"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:33.864566 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:33.864535    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj"
Sep 09 20:12:33.878702 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:33.878671    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"
Sep 09 20:12:34.147027 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.146995    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-69ggj"]
Sep 09 20:12:34.149458 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:34.149433    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f2a60f_d391_409c_a92f_a4b3ea9b9c9b.slice/crio-3fdcc5a529fcd638b994b2710ba7906b48dfaa711e9da74fd00acae6d7784c79 WatchSource:0}: Error finding container 3fdcc5a529fcd638b994b2710ba7906b48dfaa711e9da74fd00acae6d7784c79: Status 404 returned error can't find the container with id 3fdcc5a529fcd638b994b2710ba7906b48dfaa711e9da74fd00acae6d7784c79
Sep 09 20:12:34.165660 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.165636    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225"]
Sep 09 20:12:34.168958 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:34.168936    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d96ce2f_0d16_4c53_a1f0_9b4e5d363441.slice/crio-f5f9a1b770de578d9c0732304cbb3f0d04403e8d0b8755cca95505eda83c7a6d WatchSource:0}: Error finding container f5f9a1b770de578d9c0732304cbb3f0d04403e8d0b8755cca95505eda83c7a6d: Status 404 returned error can't find the container with id f5f9a1b770de578d9c0732304cbb3f0d04403e8d0b8755cca95505eda83c7a6d
Sep 09 20:12:34.844582 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.844539    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:12:34.845030 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.844634    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:12:34.847192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.847169    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdee7bc4-c3c5-4ee2-9913-f967f8649d42-metrics-tls\") pod \"dns-default-dkb4d\" (UID: \"bdee7bc4-c3c5-4ee2-9913-f967f8649d42\") " pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:12:34.847331 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.847235    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"image-registry-755fdc4cbc-ndxnf\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") " pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:12:34.854528 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.854507    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gslg8\""
Sep 09 20:12:34.862649 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.862629    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:12:34.875078 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.875058    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ftsch\""
Sep 09 20:12:34.883902 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:34.883885    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:12:35.098978 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:35.098901    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225" event={"ID":"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441","Type":"ContainerStarted","Data":"f5f9a1b770de578d9c0732304cbb3f0d04403e8d0b8755cca95505eda83c7a6d"}
Sep 09 20:12:35.100238 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:35.100212    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj" event={"ID":"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b","Type":"ContainerStarted","Data":"3fdcc5a529fcd638b994b2710ba7906b48dfaa711e9da74fd00acae6d7784c79"}
Sep 09 20:12:35.854523 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:35.854491    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkb4d"]
Sep 09 20:12:35.857171 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:35.857146    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-755fdc4cbc-ndxnf"]
Sep 09 20:12:35.857408 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:35.857389    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdee7bc4_c3c5_4ee2_9913_f967f8649d42.slice/crio-d16423e84a14318c8bfec142e77189f1695caeec23820855947e3921a6792c94 WatchSource:0}: Error finding container d16423e84a14318c8bfec142e77189f1695caeec23820855947e3921a6792c94: Status 404 returned error can't find the container with id d16423e84a14318c8bfec142e77189f1695caeec23820855947e3921a6792c94
Sep 09 20:12:35.859824 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:35.859734    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72e7028_68f7_4111_8461_15906e90decc.slice/crio-bf76e6b99b8f75550af5a1f0523ee1a53b00962f0e7387ea87315c248a093bfc WatchSource:0}: Error finding container bf76e6b99b8f75550af5a1f0523ee1a53b00962f0e7387ea87315c248a093bfc: Status 404 returned error can't find the container with id bf76e6b99b8f75550af5a1f0523ee1a53b00962f0e7387ea87315c248a093bfc
Sep 09 20:12:36.107228 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.106853    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268" event={"ID":"0d06bb0a-4fb3-4c51-b6b9-5c0120b82233","Type":"ContainerStarted","Data":"36ecb79ac29d1ca72538b4bece86591d03a271321f1cb2b292d49e093083ff63"}
Sep 09 20:12:36.107928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.107903    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:36.109739 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.109704    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkb4d" event={"ID":"bdee7bc4-c3c5-4ee2-9913-f967f8649d42","Type":"ContainerStarted","Data":"d16423e84a14318c8bfec142e77189f1695caeec23820855947e3921a6792c94"}
Sep 09 20:12:36.109944 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.109900    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268"
Sep 09 20:12:36.112561 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.112533    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf" event={"ID":"c72e7028-68f7-4111-8461-15906e90decc","Type":"ContainerStarted","Data":"5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126"}
Sep 09 20:12:36.112681 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.112581    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf" event={"ID":"c72e7028-68f7-4111-8461-15906e90decc","Type":"ContainerStarted","Data":"bf76e6b99b8f75550af5a1f0523ee1a53b00962f0e7387ea87315c248a093bfc"}
Sep 09 20:12:36.112745 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.112704    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:12:36.124776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.124728    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75c77f4f4b-cz268" podStartSLOduration=1.4923707259999999 podStartE2EDuration="7.12471381s" totalImagesPullingTime="5.632343084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:36.123939245 +0000 UTC m=+95.055022575" watchObservedRunningTime="2026-09-09 20:12:36.12471381 +0000 UTC m=+95.055797130"
Sep 09 20:12:36.160185 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:36.160111    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf" podStartSLOduration=97.160090993 podStartE2EDuration="1m37.160090993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:36.159527963 +0000 UTC m=+95.090611289" watchObservedRunningTime="2026-09-09 20:12:36.160090993 +0000 UTC m=+95.091174339"
Sep 09 20:12:37.118142 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.118088    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj" event={"ID":"21f2a60f-d391-409c-a92f-a4b3ea9b9c9b","Type":"ContainerStarted","Data":"8c2e9a3d05fc8a4f98f2c6a3e66a18339dfa8a67c28935e58f21dde9084b7a9b"}
Sep 09 20:12:37.133379 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.133330    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-69ggj" podStartSLOduration=33.876965127 podStartE2EDuration="36.133314385s" totalImagesPullingTime="2.256349258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:37.132905658 +0000 UTC m=+96.063988973" watchObservedRunningTime="2026-09-09 20:12:37.133314385 +0000 UTC m=+96.064397682"
Sep 09 20:12:37.240098 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.240050    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:37.298445 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.298353    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:37.477871 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.477787    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:37.568003 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.567966    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:37.570866 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.570839    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0de91534-394e-41b0-b58f-a199613bc5f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gtv5p\" (UID: \"0de91534-394e-41b0-b58f-a199613bc5f6\") " pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:37.578092 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.578061    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:37.650400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.650373    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"]
Sep 09 20:12:37.652643 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.652617    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:37.655053 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.654866    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 09 20:12:37.655053 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.655041    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mgrkh\""
Sep 09 20:12:37.657585 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.657478    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:37.660554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.660533    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"]
Sep 09 20:12:37.769803 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.769770    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d9a0de5-5313-4073-8fe3-dac62b357b1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-69xvj\" (UID: \"6d9a0de5-5313-4073-8fe3-dac62b357b1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:37.796295 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.796276    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gtv5p"
Sep 09 20:12:37.871153 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:37.871083    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d9a0de5-5313-4073-8fe3-dac62b357b1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-69xvj\" (UID: \"6d9a0de5-5313-4073-8fe3-dac62b357b1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:37.871281 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:37.871232    2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 09 20:12:37.871398 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:37.871384    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d9a0de5-5313-4073-8fe3-dac62b357b1d-tls-certificates podName:6d9a0de5-5313-4073-8fe3-dac62b357b1d nodeName:}" failed. No retries permitted until 2026-09-09 20:12:38.371366584 +0000 UTC m=+97.302449902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6d9a0de5-5313-4073-8fe3-dac62b357b1d-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-69xvj" (UID: "6d9a0de5-5313-4073-8fe3-dac62b357b1d") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 09 20:12:38.081243 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.081176    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gtv5p"]
Sep 09 20:12:38.084440 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:38.084417    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de91534_394e_41b0_b58f_a199613bc5f6.slice/crio-026dc4fc7c7f675a81d89d96b1e02fd92e423106d408dac01280ba8465a1c300 WatchSource:0}: Error finding container 026dc4fc7c7f675a81d89d96b1e02fd92e423106d408dac01280ba8465a1c300: Status 404 returned error can't find the container with id 026dc4fc7c7f675a81d89d96b1e02fd92e423106d408dac01280ba8465a1c300
Sep 09 20:12:38.123554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.123527    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkb4d" event={"ID":"bdee7bc4-c3c5-4ee2-9913-f967f8649d42","Type":"ContainerStarted","Data":"a31e7b59aa0816a291e8dd6101e2669e0aaa13dc54d899458ee7ccc231ff4def"}
Sep 09 20:12:38.123895 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.123583    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkb4d" event={"ID":"bdee7bc4-c3c5-4ee2-9913-f967f8649d42","Type":"ContainerStarted","Data":"57647d236b83ed5ea23856d12c3000c2b77ce3e3b29bab580521baa96cd0179d"}
Sep 09 20:12:38.123895 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.123617    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:12:38.125219 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.125192    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225" event={"ID":"7d96ce2f-0d16-4c53-a1f0-9b4e5d363441","Type":"ContainerStarted","Data":"0476dffe73ab2bc20835e6c869ae4b55490b4a093ccded7d721bfc8cb06ba528"}
Sep 09 20:12:38.126384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.126364    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gtv5p" event={"ID":"0de91534-394e-41b0-b58f-a199613bc5f6","Type":"ContainerStarted","Data":"026dc4fc7c7f675a81d89d96b1e02fd92e423106d408dac01280ba8465a1c300"}
Sep 09 20:12:38.141838 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.141801    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dkb4d" podStartSLOduration=66.306762349 podStartE2EDuration="1m8.141785317s" totalImagesPullingTime="1.835022968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:38.140262339 +0000 UTC m=+97.071345656" watchObservedRunningTime="2026-09-09 20:12:38.141785317 +0000 UTC m=+97.072868636"
Sep 09 20:12:38.157036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.156998    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9p225" podStartSLOduration=34.182405319 podStartE2EDuration="37.156983644s" totalImagesPullingTime="2.974578325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:38.156011694 +0000 UTC m=+97.087095011" watchObservedRunningTime="2026-09-09 20:12:38.156983644 +0000 UTC m=+97.088066962"
Sep 09 20:12:38.274012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.273975    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:38.375447 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.375417    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d9a0de5-5313-4073-8fe3-dac62b357b1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-69xvj\" (UID: \"6d9a0de5-5313-4073-8fe3-dac62b357b1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:38.377674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.377656    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d9a0de5-5313-4073-8fe3-dac62b357b1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-69xvj\" (UID: \"6d9a0de5-5313-4073-8fe3-dac62b357b1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:38.564963 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.564931    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:38.828674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:38.828650    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"]
Sep 09 20:12:38.837214 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:38.837186    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9a0de5_5313_4073_8fe3_dac62b357b1d.slice/crio-0626064f01742d94251051096c037ae226e8a9c3d1a6a461c4a8ee9a425c326f WatchSource:0}: Error finding container 0626064f01742d94251051096c037ae226e8a9c3d1a6a461c4a8ee9a425c326f: Status 404 returned error can't find the container with id 0626064f01742d94251051096c037ae226e8a9c3d1a6a461c4a8ee9a425c326f
Sep 09 20:12:39.131622 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:39.131588    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gtv5p" event={"ID":"0de91534-394e-41b0-b58f-a199613bc5f6","Type":"ContainerStarted","Data":"3d902f23063337f69290c74a0494debd0f5327e49dd52e85ad49b280cd4f47ac"}
Sep 09 20:12:39.132026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:39.131655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gtv5p" event={"ID":"0de91534-394e-41b0-b58f-a199613bc5f6","Type":"ContainerStarted","Data":"b7ca1531005c4a70fd8ee3d920c32fae9abda385956e7c17b5ddbadf07c1d0e4"}
Sep 09 20:12:39.132754 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:39.132729    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj" event={"ID":"6d9a0de5-5313-4073-8fe3-dac62b357b1d","Type":"ContainerStarted","Data":"0626064f01742d94251051096c037ae226e8a9c3d1a6a461c4a8ee9a425c326f"}
Sep 09 20:12:41.143019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.142983    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gtv5p" event={"ID":"0de91534-394e-41b0-b58f-a199613bc5f6","Type":"ContainerStarted","Data":"1a439099d43db0ceca75af230733d075bc82290e7f9299e8eccef00b21cdc61d"}
Sep 09 20:12:41.144707 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.144675    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj" event={"ID":"6d9a0de5-5313-4073-8fe3-dac62b357b1d","Type":"ContainerStarted","Data":"f74cf712ef53e5e342e959d2d9206d6cc6fb460fce236292d86705075d538466"}
Sep 09 20:12:41.144823 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.144730    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:41.149612 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.149589    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj"
Sep 09 20:12:41.163630 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.163590    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gtv5p" podStartSLOduration=33.946105004 podStartE2EDuration="36.16357561s" totalImagesPullingTime="2.217470606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:12:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:41.163098254 +0000 UTC m=+100.094181574" watchObservedRunningTime="2026-09-09 20:12:41.16357561 +0000 UTC m=+100.094658928"
Sep 09 20:12:41.179576 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.179526    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-69xvj" podStartSLOduration=2.602110255 podStartE2EDuration="4.179512604s" totalImagesPullingTime="1.577402349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:41.178581807 +0000 UTC m=+100.109665126" watchObservedRunningTime="2026-09-09 20:12:41.179512604 +0000 UTC m=+100.110595922"
Sep 09 20:12:41.213539 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.213516    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:41.678187 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.678148    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:41.723545 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.723514    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pq466"]
Sep 09 20:12:41.728556 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.728539    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.730931 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.730903    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 09 20:12:41.731037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.730979    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 09 20:12:41.731968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.731921    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qhcmv\""
Sep 09 20:12:41.731968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.731932    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 20:12:41.735512 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.735483    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pq466"]
Sep 09 20:12:41.738472 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.738450    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:41.801503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.801476    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f2aed8-212b-4623-bd72-218e131a26c3-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.801615 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.801518    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pchm4\" (UniqueName: \"kubernetes.io/projected/d3f2aed8-212b-4623-bd72-218e131a26c3-kube-api-access-pchm4\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.801615 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.801545    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.801615 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.801592    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.902328 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.902304    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pchm4\" (UniqueName: \"kubernetes.io/projected/d3f2aed8-212b-4623-bd72-218e131a26c3-kube-api-access-pchm4\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.902444 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.902335    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.902444 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.902355    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.902444 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.902392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f2aed8-212b-4623-bd72-218e131a26c3-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.902599 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:41.902519    2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 09 20:12:41.902599 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:41.902589    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-tls podName:d3f2aed8-212b-4623-bd72-218e131a26c3 nodeName:}" failed. No retries permitted until 2026-09-09 20:12:42.402566762 +0000 UTC m=+101.333650059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-pq466" (UID: "d3f2aed8-212b-4623-bd72-218e131a26c3") : secret "prometheus-operator-tls" not found
Sep 09 20:12:41.903002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.902986    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f2aed8-212b-4623-bd72-218e131a26c3-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.904758 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.904735    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:41.911235 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:41.911215    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pchm4\" (UniqueName: \"kubernetes.io/projected/d3f2aed8-212b-4623-bd72-218e131a26c3-kube-api-access-pchm4\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:42.407077 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:42.407048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:42.409287 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:42.409269    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d3f2aed8-212b-4623-bd72-218e131a26c3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pq466\" (UID: \"d3f2aed8-212b-4623-bd72-218e131a26c3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:42.422451 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:42.422428    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:42.638828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:42.638797    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466"
Sep 09 20:12:42.904240 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:42.904206    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pq466"]
Sep 09 20:12:42.908147 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:42.908124    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f2aed8_212b_4623_bd72_218e131a26c3.slice/crio-8f25c8645e9d4a3a99a0c45401e7255e5a3cb7b53a6e5013678f90d38f06bdf6 WatchSource:0}: Error finding container 8f25c8645e9d4a3a99a0c45401e7255e5a3cb7b53a6e5013678f90d38f06bdf6: Status 404 returned error can't find the container with id 8f25c8645e9d4a3a99a0c45401e7255e5a3cb7b53a6e5013678f90d38f06bdf6
Sep 09 20:12:43.147452 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:43.147415    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:43.151906 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:43.151878    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466" event={"ID":"d3f2aed8-212b-4623-bd72-218e131a26c3","Type":"ContainerStarted","Data":"8f25c8645e9d4a3a99a0c45401e7255e5a3cb7b53a6e5013678f90d38f06bdf6"}
Sep 09 20:12:43.248846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:43.248814    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:44.612873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:44.612845    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:45.161050 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:45.161017    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466" event={"ID":"d3f2aed8-212b-4623-bd72-218e131a26c3","Type":"ContainerStarted","Data":"441fc42c5369f57c8ff9861715939d6c7eccaf2ab54c7050a688609715ed3edd"}
Sep 09 20:12:45.161222 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:45.161068    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466" event={"ID":"d3f2aed8-212b-4623-bd72-218e131a26c3","Type":"ContainerStarted","Data":"aa849985d0d0498a625d468b465161f68c20b492b96a7494823b6237dbba1f4e"}
Sep 09 20:12:45.183437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:45.183389    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-pq466" podStartSLOduration=2.775386963 podStartE2EDuration="4.183371811s" totalImagesPullingTime="1.407984848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:45.181770327 +0000 UTC m=+104.112853644" watchObservedRunningTime="2026-09-09 20:12:45.183371811 +0000 UTC m=+104.114455130"
Sep 09 20:12:46.836628 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:46.836592    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.111506 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.111466    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.193269 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.193229    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l87wp"]
Sep 09 20:12:47.196881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.196859    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.199390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.199366    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 20:12:47.199510 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.199389    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 20:12:47.199510 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.199406    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9nvqc\""
Sep 09 20:12:47.199712 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.199694    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 20:12:47.230685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.230652    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.319470 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.319438    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.350700 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350659    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-tls\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.350881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-wtmp\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.350881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350774    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-root\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.350881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350844    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mml4g\" (UniqueName: \"kubernetes.io/projected/638e9f45-54c2-4de4-bdd6-ca54251673de-kube-api-access-mml4g\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.351038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350907    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-accelerators-collector-config\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.351038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350945    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-sys\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.351038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.350969    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/638e9f45-54c2-4de4-bdd6-ca54251673de-metrics-client-ca\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.351038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.351031    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-textfile\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.351236 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.351080    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452132 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-tls\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452132 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452083    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-wtmp\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452132 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452105    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-root\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452170    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-root\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mml4g\" (UniqueName: \"kubernetes.io/projected/638e9f45-54c2-4de4-bdd6-ca54251673de-kube-api-access-mml4g\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452308    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-accelerators-collector-config\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452273    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-wtmp\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452352    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-sys\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452395    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/638e9f45-54c2-4de4-bdd6-ca54251673de-metrics-client-ca\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452402    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/638e9f45-54c2-4de4-bdd6-ca54251673de-sys\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452754 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452462    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-textfile\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452754 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452506    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452856 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452823    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-textfile\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.452973 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452952    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-accelerators-collector-config\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.453017 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.452967    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/638e9f45-54c2-4de4-bdd6-ca54251673de-metrics-client-ca\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.454606 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.454584    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-tls\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.454733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.454713    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/638e9f45-54c2-4de4-bdd6-ca54251673de-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.460577 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.460554    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mml4g\" (UniqueName: \"kubernetes.io/projected/638e9f45-54c2-4de4-bdd6-ca54251673de-kube-api-access-mml4g\") pod \"node-exporter-l87wp\" (UID: \"638e9f45-54c2-4de4-bdd6-ca54251673de\") " pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.495678 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.495647    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.506564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.506540    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l87wp"
Sep 09 20:12:47.520453 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:47.520426    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638e9f45_54c2_4de4_bdd6_ca54251673de.slice/crio-5bf6bf7c644e7fd24aef16df79fa9ce615811c92645b6002e17aa28261e4de70 WatchSource:0}: Error finding container 5bf6bf7c644e7fd24aef16df79fa9ce615811c92645b6002e17aa28261e4de70: Status 404 returned error can't find the container with id 5bf6bf7c644e7fd24aef16df79fa9ce615811c92645b6002e17aa28261e4de70
Sep 09 20:12:47.596377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.596343    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.696437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.696398    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:47.781682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:47.781611    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.017078 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.017024    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.119721 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.119680    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.135939 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.135913    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dkb4d"
Sep 09 20:12:48.176358 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.176321    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l87wp" event={"ID":"638e9f45-54c2-4de4-bdd6-ca54251673de","Type":"ContainerStarted","Data":"5bf6bf7c644e7fd24aef16df79fa9ce615811c92645b6002e17aa28261e4de70"}
Sep 09 20:12:48.214488 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.214454    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.340412 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.340385    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.438654 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.438563    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.538669 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.538644    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.640958 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.640921    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:48.991139 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:48.991109    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:49.090748 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.090711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:49.170921 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.170889    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-55797c89d5-tz4p5"]
Sep 09 20:12:49.174234 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.174218    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.176839    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-krj69\""
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.177109    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.177173    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-20vpekbqub6co\""
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.177209    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.177110    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.177110    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 09 20:12:49.177757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.177550    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 09 20:12:49.184365 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.184338    2573 generic.go:356] "Generic (PLEG): container finished" podID="638e9f45-54c2-4de4-bdd6-ca54251673de" containerID="ffec1a2d51e0470f570ecf26e4b00cc01df5d49912e1792ef78f4b1e83250a2b" exitCode=0
Sep 09 20:12:49.184490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.184427    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l87wp" event={"ID":"638e9f45-54c2-4de4-bdd6-ca54251673de","Type":"ContainerDied","Data":"ffec1a2d51e0470f570ecf26e4b00cc01df5d49912e1792ef78f4b1e83250a2b"}
Sep 09 20:12:49.186232 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.186212    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55797c89d5-tz4p5"]
Sep 09 20:12:49.268197 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268166    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268335 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268222    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268395 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268350    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268441 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268410    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-grpc-tls\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268492    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268578 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268530    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5hh9\" (UniqueName: \"kubernetes.io/projected/f0e42510-ddf9-40ce-914a-db8ad52e574a-kube-api-access-j5hh9\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268578 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268550    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0e42510-ddf9-40ce-914a-db8ad52e574a-metrics-client-ca\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.268669 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.268606    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-tls\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369598 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369566    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-grpc-tls\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369618    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369644    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5hh9\" (UniqueName: \"kubernetes.io/projected/f0e42510-ddf9-40ce-914a-db8ad52e574a-kube-api-access-j5hh9\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369660    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0e42510-ddf9-40ce-914a-db8ad52e574a-metrics-client-ca\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369698    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-tls\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369733    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.369775 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369767    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.370079 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.369799    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.370570 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.370534    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0e42510-ddf9-40ce-914a-db8ad52e574a-metrics-client-ca\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.372515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.372465    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.372620 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.372548    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-grpc-tls\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.372620 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.372600    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.372711 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.372684    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.372836 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.372821    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.372869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.372837    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f0e42510-ddf9-40ce-914a-db8ad52e574a-secret-thanos-querier-tls\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.378536 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.378514    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5hh9\" (UniqueName: \"kubernetes.io/projected/f0e42510-ddf9-40ce-914a-db8ad52e574a-kube-api-access-j5hh9\") pod \"thanos-querier-55797c89d5-tz4p5\" (UID: \"f0e42510-ddf9-40ce-914a-db8ad52e574a\") " pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.485924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.485890    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:49.764317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.764045    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55797c89d5-tz4p5"]
Sep 09 20:12:49.766690 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:49.766659    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e42510_ddf9_40ce_914a_db8ad52e574a.slice/crio-07db505b6ec65e19ad7d4f3835945fc7ab2534941aba37f8335411a3a310964f WatchSource:0}: Error finding container 07db505b6ec65e19ad7d4f3835945fc7ab2534941aba37f8335411a3a310964f: Status 404 returned error can't find the container with id 07db505b6ec65e19ad7d4f3835945fc7ab2534941aba37f8335411a3a310964f
Sep 09 20:12:49.933869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:49.933829    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.034415 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.034335    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.135219 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.135187    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.190494 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.190460    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"07db505b6ec65e19ad7d4f3835945fc7ab2534941aba37f8335411a3a310964f"}
Sep 09 20:12:50.192930 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.192900    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l87wp" event={"ID":"638e9f45-54c2-4de4-bdd6-ca54251673de","Type":"ContainerStarted","Data":"f9afa53d5192611728d807a21bd87e8341994ee1f8346cbaaeae7845bc6eaece"}
Sep 09 20:12:50.193067 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.192952    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l87wp" event={"ID":"638e9f45-54c2-4de4-bdd6-ca54251673de","Type":"ContainerStarted","Data":"a6e51dab2e044583e668a69f951415e7910124ae841683584ba1db18bfb4993a"}
Sep 09 20:12:50.220291 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.220211    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l87wp" podStartSLOduration=2.178048607 podStartE2EDuration="3.220193277s" totalImagesPullingTime="980.75427ms" totalInitContainerRuntime="61.3904ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:50.219039364 +0000 UTC m=+109.150122685" watchObservedRunningTime="2026-09-09 20:12:50.220193277 +0000 UTC m=+109.151276596"
Sep 09 20:12:50.229586 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.229541    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.343416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.343321    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.439007 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.438963    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.574220 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.574175    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:50.656293 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:50.656233    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:51.016785 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.016680    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:51.569338 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.569314    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6646dfc98-csdnq"]
Sep 09 20:12:51.573447 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.573431    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.575896 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.575873    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 20:12:51.575896 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.575887    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 20:12:51.576129 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.576114    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 20:12:51.577131 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.577114    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hxvdb\""
Sep 09 20:12:51.577442 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.577152    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 20:12:51.577442 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.577320    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2i2birug90pr8\""
Sep 09 20:12:51.582694 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.582669    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6646dfc98-csdnq"]
Sep 09 20:12:51.692067 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692043    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-client-ca-bundle\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.692221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ecf9620c-dd13-4c5a-a61b-04bd353c9877-audit-log\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.692221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692096    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf9620c-dd13-4c5a-a61b-04bd353c9877-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.692221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692116    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-secret-metrics-server-client-certs\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.692221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692134    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-secret-metrics-server-tls\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.692221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692155    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlbj8\" (UniqueName: \"kubernetes.io/projected/ecf9620c-dd13-4c5a-a61b-04bd353c9877-kube-api-access-zlbj8\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.692221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.692206    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ecf9620c-dd13-4c5a-a61b-04bd353c9877-metrics-server-audit-profiles\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793379    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-client-ca-bundle\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793414    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ecf9620c-dd13-4c5a-a61b-04bd353c9877-audit-log\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf9620c-dd13-4c5a-a61b-04bd353c9877-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-secret-metrics-server-client-certs\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-secret-metrics-server-tls\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793502    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlbj8\" (UniqueName: \"kubernetes.io/projected/ecf9620c-dd13-4c5a-a61b-04bd353c9877-kube-api-access-zlbj8\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793537    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ecf9620c-dd13-4c5a-a61b-04bd353c9877-metrics-server-audit-profiles\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.793944 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.793905    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ecf9620c-dd13-4c5a-a61b-04bd353c9877-audit-log\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.794325 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.794302    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecf9620c-dd13-4c5a-a61b-04bd353c9877-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.794481 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.794463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ecf9620c-dd13-4c5a-a61b-04bd353c9877-metrics-server-audit-profiles\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.796101 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.796079    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-client-ca-bundle\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.796183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.796083    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-secret-metrics-server-client-certs\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.796183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.796113    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ecf9620c-dd13-4c5a-a61b-04bd353c9877-secret-metrics-server-tls\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.807737 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.806696    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlbj8\" (UniqueName: \"kubernetes.io/projected/ecf9620c-dd13-4c5a-a61b-04bd353c9877-kube-api-access-zlbj8\") pod \"metrics-server-6646dfc98-csdnq\" (UID: \"ecf9620c-dd13-4c5a-a61b-04bd353c9877\") " pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.834592 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.834558    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:51.850541 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.850514    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"]
Sep 09 20:12:51.854934 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.854916    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:51.857105 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.857082    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 20:12:51.857192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.857087    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mb5js\""
Sep 09 20:12:51.862926 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.862901    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"]
Sep 09 20:12:51.886619 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.886593    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:12:51.928320 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.928281    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-755fdc4cbc-ndxnf"]
Sep 09 20:12:51.933963 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.933936    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:12:51.934572 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.934542    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:51.995714 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.995688    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75w5k\" (UniqueName: \"kubernetes.io/projected/48e41d1b-87df-49f6-9908-e2bc20f8210e-kube-api-access-75w5k\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:51.995827 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:51.995762    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48e41d1b-87df-49f6-9908-e2bc20f8210e-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:52.045748 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.045687    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:52.096477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.096452    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48e41d1b-87df-49f6-9908-e2bc20f8210e-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:52.096587 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.096512    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75w5k\" (UniqueName: \"kubernetes.io/projected/48e41d1b-87df-49f6-9908-e2bc20f8210e-kube-api-access-75w5k\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:52.096643 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:52.096603    2573 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 20:12:52.096694 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:12:52.096671    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48e41d1b-87df-49f6-9908-e2bc20f8210e-monitoring-plugin-cert podName:48e41d1b-87df-49f6-9908-e2bc20f8210e nodeName:}" failed. No retries permitted until 2026-09-09 20:12:52.596652573 +0000 UTC m=+111.527735881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/48e41d1b-87df-49f6-9908-e2bc20f8210e-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-bnlxx" (UID: "48e41d1b-87df-49f6-9908-e2bc20f8210e") : secret "monitoring-plugin-cert" not found
Sep 09 20:12:52.106423 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.106402    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75w5k\" (UniqueName: \"kubernetes.io/projected/48e41d1b-87df-49f6-9908-e2bc20f8210e-kube-api-access-75w5k\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:52.145882 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.145854    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:52.163131 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.163107    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6646dfc98-csdnq"]
Sep 09 20:12:52.166368 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:52.166349    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf9620c_dd13_4c5a_a61b_04bd353c9877.slice/crio-dbb88977f22ab9d21b6ea4b9dd9dad64cfb82c3043554e53df23da9fc456041f WatchSource:0}: Error finding container dbb88977f22ab9d21b6ea4b9dd9dad64cfb82c3043554e53df23da9fc456041f: Status 404 returned error can't find the container with id dbb88977f22ab9d21b6ea4b9dd9dad64cfb82c3043554e53df23da9fc456041f
Sep 09 20:12:52.201980 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.201954    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"09c56ec80e32a480bdbcf02540de616a71e049702a91f6dd287e6f3a9c4ef09d"}
Sep 09 20:12:52.202072 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.202014    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"c8294f9eac31c38b43fe353b629137f2507c81484485bdf4d145b5dce261245b"}
Sep 09 20:12:52.202072 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.202053    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"055b1817799af63474cd37ca8c9d1b5900873fa6bd907b5b20d6f85b759b3786"}
Sep 09 20:12:52.203159 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.203140    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq" event={"ID":"ecf9620c-dd13-4c5a-a61b-04bd353c9877","Type":"ContainerStarted","Data":"dbb88977f22ab9d21b6ea4b9dd9dad64cfb82c3043554e53df23da9fc456041f"}
Sep 09 20:12:52.245377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.245355    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:52.346184 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.346125    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:52.601545 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.601516    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48e41d1b-87df-49f6-9908-e2bc20f8210e-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:52.604881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.604844    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/48e41d1b-87df-49f6-9908-e2bc20f8210e-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-bnlxx\" (UID: \"48e41d1b-87df-49f6-9908-e2bc20f8210e\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:52.764796 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:52.764775    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:53.037026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.036991    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"]
Sep 09 20:12:53.040850 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:53.040823    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e41d1b_87df_49f6_9908_e2bc20f8210e.slice/crio-ae15b007d771e50afcb1019ee05a41eed8bb5b72cedc29dfc88a8e4c549e41b9 WatchSource:0}: Error finding container ae15b007d771e50afcb1019ee05a41eed8bb5b72cedc29dfc88a8e4c549e41b9: Status 404 returned error can't find the container with id ae15b007d771e50afcb1019ee05a41eed8bb5b72cedc29dfc88a8e4c549e41b9
Sep 09 20:12:53.210171 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.210084    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx" event={"ID":"48e41d1b-87df-49f6-9908-e2bc20f8210e","Type":"ContainerStarted","Data":"ae15b007d771e50afcb1019ee05a41eed8bb5b72cedc29dfc88a8e4c549e41b9"}
Sep 09 20:12:53.214733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.214405    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"4120f2cc7c926025a2b0b1889d983e232ea948fb7490d1c00540d2728204d03c"}
Sep 09 20:12:53.214733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.214484    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"85ac6e13adeee8f4654e349b416e36a5af8f28aeb37737568555f69465acd55c"}
Sep 09 20:12:53.214733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.214523    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" event={"ID":"f0e42510-ddf9-40ce-914a-db8ad52e574a","Type":"ContainerStarted","Data":"c416c1d00bea69b7ab25af4eb932d790873c9d871809f4f96c513d938bc4e172"}
Sep 09 20:12:53.220954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.215574    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:12:53.246961 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.246910    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5" podStartSLOduration=1.479451419 podStartE2EDuration="4.246889279s" totalImagesPullingTime="2.76743786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:53.245927632 +0000 UTC m=+112.177010953" watchObservedRunningTime="2026-09-09 20:12:53.246889279 +0000 UTC m=+112.177972599"
Sep 09 20:12:53.389716 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.389679    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:12:53.392977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.392395    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:53.394596 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.394572    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.398882 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.398861    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 20:12:53.399005 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.398953    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qq89s\""
Sep 09 20:12:53.399005 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.398987    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 20:12:53.399128 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399114    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 20:12:53.399213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399195    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 20:12:53.399309 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399290    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 20:12:53.399383 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399356    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 20:12:53.399446 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399387    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cto4gt88risbs\""
Sep 09 20:12:53.399557 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399538    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 20:12:53.399625 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399592    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 20:12:53.399749 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399731    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 20:12:53.399814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399796    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 20:12:53.399881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399865    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 20:12:53.399913 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.399865    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 20:12:53.400216 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.400192    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 20:12:53.402006 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.401983    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 20:12:53.411094 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.411075    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:12:53.511359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511267    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511326    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-config\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511357    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511376    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511402    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511449    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511542    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511587    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shb6\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-kube-api-access-2shb6\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511616    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511670    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-web-config\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511697    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511721    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511741    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-config-out\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511803    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511843    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511908    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511953    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.511982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.511978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.613288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.613227    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shb6\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-kube-api-access-2shb6\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.613738 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.613379    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.613426    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-web-config\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614012    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-config-out\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614100    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614130    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614176    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614204    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614283    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614322    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614366    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-config\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614387    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614416    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614443    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614469    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.614937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614493    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.615852 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.614524    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.615852 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.615611    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.617433 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.617403    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.617787 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.617762    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.618096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.618073    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.618173 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.618150    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.619100 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.619074    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.619391 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.619366    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-web-config\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.619551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.619526    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.619761 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.619738    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.620395 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.620368    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.620673 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.620651    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.622518 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.622496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.622877 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.622849    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.623689 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.623668    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.624150 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.624131    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.624237 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.624216    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shb6\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-kube-api-access-2shb6\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.624785 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.624729    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.625533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.625453    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-config-out\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.626236 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.626213    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-config\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.626629 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.626607    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.707142 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.707117    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:12:53.994995 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:53.994971    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:12:53.997792 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:12:53.997757    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18fabdbc_0648_42f7_8acc_30187ae19e69.slice/crio-1ad293fa691ce7d8c0fa1a737678564945db8e71000f91e7c831557825f105b9 WatchSource:0}: Error finding container 1ad293fa691ce7d8c0fa1a737678564945db8e71000f91e7c831557825f105b9: Status 404 returned error can't find the container with id 1ad293fa691ce7d8c0fa1a737678564945db8e71000f91e7c831557825f105b9
Sep 09 20:12:54.221159 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:54.221126    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq" event={"ID":"ecf9620c-dd13-4c5a-a61b-04bd353c9877","Type":"ContainerStarted","Data":"edb60a88f11db2743071ddb1e6d374e2e10360a4abe959502dccc194c7186eca"}
Sep 09 20:12:54.222641 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:54.222614    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"1ad293fa691ce7d8c0fa1a737678564945db8e71000f91e7c831557825f105b9"}
Sep 09 20:12:55.035162 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.035130    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:55.135786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.135753    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:55.220110 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.220077    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:55.227411 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.227377    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx" event={"ID":"48e41d1b-87df-49f6-9908-e2bc20f8210e","Type":"ContainerStarted","Data":"870c5672ca621af34a6e6edba0ca67d84f624f68ac2af255778f2d978f6af51c"}
Sep 09 20:12:55.227564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.227547    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:55.232543 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.232523    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx"
Sep 09 20:12:55.245184 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.245140    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq" podStartSLOduration=2.706495872 podStartE2EDuration="4.245127012s" totalImagesPullingTime="1.53863114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:54.241481851 +0000 UTC m=+113.172565171" watchObservedRunningTime="2026-09-09 20:12:55.245127012 +0000 UTC m=+114.176210330"
Sep 09 20:12:55.246093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:55.246056    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-bnlxx" podStartSLOduration=2.748020136 podStartE2EDuration="4.246043516s" totalImagesPullingTime="1.49802338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:12:55.243937349 +0000 UTC m=+114.175020669" watchObservedRunningTime="2026-09-09 20:12:55.246043516 +0000 UTC m=+114.177126836"
Sep 09 20:12:56.232119 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:56.232084    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040" exitCode=0
Sep 09 20:12:56.232544 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:56.232170    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040"}
Sep 09 20:12:57.273940 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:57.273898    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:57.353662 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:57.353612    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:12:59.230897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:12:59.230870    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-55797c89d5-tz4p5"
Sep 09 20:13:00.255381 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.255183    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb"}
Sep 09 20:13:00.255381 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.255284    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4"}
Sep 09 20:13:00.255381 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.255326    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721"}
Sep 09 20:13:00.255381 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.255378    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2"}
Sep 09 20:13:00.255910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.255416    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe"}
Sep 09 20:13:00.255910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.255453    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerStarted","Data":"d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9"}
Sep 09 20:13:00.286542 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:00.286490    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.832461497 podStartE2EDuration="7.286473396s" totalImagesPullingTime="4.432710648s" totalInitContainerRuntime="21.301251ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:12:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:13:00.284585401 +0000 UTC m=+119.215668724" watchObservedRunningTime="2026-09-09 20:13:00.286473396 +0000 UTC m=+119.217556715"
Sep 09 20:13:01.142354 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:01.142303    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:01.205319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:01.205286    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:03.707660 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:03.707630    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:13:05.132945 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:05.132893    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:05.194823 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:05.194791    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:05.570849 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:05.570770    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:05.671802 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:05.671764    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:06.281926 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:06.281895    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:06.357982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:06.357949    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:09.779085 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:09.779042    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:09.881588 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:09.881553    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:10.354494 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.354462    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:10.359604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.359583    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:13:10.362133 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.362114    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a080ef-623f-4b93-be6c-2b4355b2afc2-metrics-certs\") pod \"network-metrics-daemon-5ghhl\" (UID: \"80a080ef-623f-4b93-be6c-2b4355b2afc2\") " pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:13:10.419080 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.419048    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:10.452631 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.452602    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4t9vt\""
Sep 09 20:13:10.460633 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.460617    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ghhl"
Sep 09 20:13:10.742640 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.742536    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:10.798424 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:10.798375    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5ghhl"]
Sep 09 20:13:10.800541 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:13:10.800513    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a080ef_623f_4b93_be6c_2b4355b2afc2.slice/crio-067850e87340a8463ffbc6b918b08004eb22090d9c648e037012d5d82cd63bf5 WatchSource:0}: Error finding container 067850e87340a8463ffbc6b918b08004eb22090d9c648e037012d5d82cd63bf5: Status 404 returned error can't find the container with id 067850e87340a8463ffbc6b918b08004eb22090d9c648e037012d5d82cd63bf5
Sep 09 20:13:11.300207 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:11.300171    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5ghhl" event={"ID":"80a080ef-623f-4b93-be6c-2b4355b2afc2","Type":"ContainerStarted","Data":"067850e87340a8463ffbc6b918b08004eb22090d9c648e037012d5d82cd63bf5"}
Sep 09 20:13:11.887561 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:11.887526    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:13:11.887950 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:11.887640    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:13:12.306147 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:12.306111    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5ghhl" event={"ID":"80a080ef-623f-4b93-be6c-2b4355b2afc2","Type":"ContainerStarted","Data":"32443ff609848f1f10ddff17e49d0a67e7074f202d1bb2412d7f65af496ee9de"}
Sep 09 20:13:13.311589 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:13.311555    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5ghhl" event={"ID":"80a080ef-623f-4b93-be6c-2b4355b2afc2","Type":"ContainerStarted","Data":"54a94bebff5e153fd2125b29af334b3621875c94081f040824716d37e873fac3"}
Sep 09 20:13:13.328984 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:13.328930    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5ghhl" podStartSLOduration=129.982759109 podStartE2EDuration="2m11.328911578s" totalImagesPullingTime="1.346152469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:13:13.327121241 +0000 UTC m=+132.258204560" watchObservedRunningTime="2026-09-09 20:13:13.328911578 +0000 UTC m=+132.259994943"
Sep 09 20:13:13.337530 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:13.337496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:13.417524 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:13.417488    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:13.512414 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:13.512380    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:16.956094 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:16.956051    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf" podUID="c72e7028-68f7-4111-8461-15906e90decc" containerName="registry" containerID="cri-o://5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126" gracePeriod=30
Sep 09 20:13:17.201234 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.201213    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:13:17.328002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.327902    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.327946    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-registry-certificates\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.327990    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-bound-sa-token\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328296 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328052    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqltz\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-kube-api-access-pqltz\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328296 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328089    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-installation-pull-secrets\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328296 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328116    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c72e7028-68f7-4111-8461-15906e90decc-ca-trust-extracted\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328296 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328151    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-image-registry-private-configuration\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328296 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328187    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-trusted-ca\") pod \"c72e7028-68f7-4111-8461-15906e90decc\" (UID: \"c72e7028-68f7-4111-8461-15906e90decc\") "
Sep 09 20:13:17.328820 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328785    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:13:17.328930 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.328837    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:13:17.331462 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.331417    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:13:17.331884 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.331840    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:13:17.332455 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.332424    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:13:17.333355 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.333229    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:13:17.333611 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.333534    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-kube-api-access-pqltz" (OuterVolumeSpecName: "kube-api-access-pqltz") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "kube-api-access-pqltz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:13:17.335537 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.335511    2573 generic.go:356] "Generic (PLEG): container finished" podID="c72e7028-68f7-4111-8461-15906e90decc" containerID="5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126" exitCode=0
Sep 09 20:13:17.335828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.335557    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf" event={"ID":"c72e7028-68f7-4111-8461-15906e90decc","Type":"ContainerDied","Data":"5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126"}
Sep 09 20:13:17.335828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.335590    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf"
Sep 09 20:13:17.335828 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.335605    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755fdc4cbc-ndxnf" event={"ID":"c72e7028-68f7-4111-8461-15906e90decc","Type":"ContainerDied","Data":"bf76e6b99b8f75550af5a1f0523ee1a53b00962f0e7387ea87315c248a093bfc"}
Sep 09 20:13:17.336683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.336659    2573 scope.go:117] "RemoveContainer" containerID="5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126"
Sep 09 20:13:17.341573 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.341545    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c72e7028-68f7-4111-8461-15906e90decc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c72e7028-68f7-4111-8461-15906e90decc" (UID: "c72e7028-68f7-4111-8461-15906e90decc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:13:17.353923 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.353905    2573 scope.go:117] "RemoveContainer" containerID="5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126"
Sep 09 20:13:17.354222 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:13:17.354201    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126\": container with ID starting with 5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126 not found: ID does not exist" containerID="5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126"
Sep 09 20:13:17.354318 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.354235    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126"} err="failed to get container status \"5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126\": rpc error: code = NotFound desc = could not find container \"5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126\": container with ID starting with 5fc3638ea6a4c00774c49691e12f4bbb6ccacf8769863ec79de27ce79cd1b126 not found: ID does not exist"
Sep 09 20:13:17.429261 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429212    2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-installation-pull-secrets\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429281    2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c72e7028-68f7-4111-8461-15906e90decc-ca-trust-extracted\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429296    2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c72e7028-68f7-4111-8461-15906e90decc-image-registry-private-configuration\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429308    2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-trusted-ca\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429319    2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-registry-tls\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429330    2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c72e7028-68f7-4111-8461-15906e90decc-registry-certificates\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429342    2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-bound-sa-token\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.429416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.429354    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqltz\" (UniqueName: \"kubernetes.io/projected/c72e7028-68f7-4111-8461-15906e90decc-kube-api-access-pqltz\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:13:17.438117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.438089    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:17.578269 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.578153    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:17.641289 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.641233    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:17.660285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.660242    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-755fdc4cbc-ndxnf"]
Sep 09 20:13:17.664183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:17.664159    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-755fdc4cbc-ndxnf"]
Sep 09 20:13:19.641175 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:19.641142    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c72e7028-68f7-4111-8461-15906e90decc" path="/var/lib/kubelet/pods/c72e7028-68f7-4111-8461-15906e90decc/volumes"
Sep 09 20:13:20.565480 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:20.565450    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:20.624469 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:20.624444    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:31.892676 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:31.892647    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:13:31.896403 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:31.896382    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6646dfc98-csdnq"
Sep 09 20:13:35.411596 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:35.411558    2573 generic.go:356] "Generic (PLEG): container finished" podID="0dd432c4-c58f-4523-a3a0-6ca0b6e27366" containerID="69aee83df5048b49edbb9d6e062eb3a967c753b74b18ace3b85e4f67ce5ec3df" exitCode=0
Sep 09 20:13:35.412103 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:35.411637    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq" event={"ID":"0dd432c4-c58f-4523-a3a0-6ca0b6e27366","Type":"ContainerDied","Data":"69aee83df5048b49edbb9d6e062eb3a967c753b74b18ace3b85e4f67ce5ec3df"}
Sep 09 20:13:35.412103 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:35.412091    2573 scope.go:117] "RemoveContainer" containerID="69aee83df5048b49edbb9d6e062eb3a967c753b74b18ace3b85e4f67ce5ec3df"
Sep 09 20:13:35.573107 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:35.573074    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:35.635672 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:35.635634    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:13:36.417668 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:36.417625    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-dm4lq" event={"ID":"0dd432c4-c58f-4523-a3a0-6ca0b6e27366","Type":"ContainerStarted","Data":"92a5bb7a2dcec02beb7b529b670fdca1333a9beb7bb8445c2249a08a96559a9f"}
Sep 09 20:13:53.708201 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:53.708162    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:13:53.727601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:53.727572    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:13:54.506090 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:13:54.506063    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:06.387382 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:06.387345    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:06.596794 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:06.596754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:11.804905 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.804874    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:14:11.805497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.805181    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="prometheus" containerID="cri-o://d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9" gracePeriod=600
Sep 09 20:14:11.805497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.805227    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-web" containerID="cri-o://32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721" gracePeriod=600
Sep 09 20:14:11.805497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.805241    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-thanos" containerID="cri-o://edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb" gracePeriod=600
Sep 09 20:14:11.805497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.805279    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="thanos-sidecar" containerID="cri-o://55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2" gracePeriod=600
Sep 09 20:14:11.805497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.805268    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="config-reloader" containerID="cri-o://484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe" gracePeriod=600
Sep 09 20:14:11.805774 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:11.805522    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy" containerID="cri-o://e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4" gracePeriod=600
Sep 09 20:14:12.563163 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563129    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb" exitCode=0
Sep 09 20:14:12.563163 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563154    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4" exitCode=0
Sep 09 20:14:12.563163 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563160    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2" exitCode=0
Sep 09 20:14:12.563163 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563166    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe" exitCode=0
Sep 09 20:14:12.563163 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563171    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9" exitCode=0
Sep 09 20:14:12.563477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563207    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb"}
Sep 09 20:14:12.563477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563287    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4"}
Sep 09 20:14:12.563477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563316    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2"}
Sep 09 20:14:12.563477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563341    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe"}
Sep 09 20:14:12.563477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:12.563363    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9"}
Sep 09 20:14:13.044245 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.044216    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.091802 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091773    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-db\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.091974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091811    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-grpc-tls\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.091974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091839    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-serving-certs-ca-bundle\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.091974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091864    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-kubelet-serving-ca-bundle\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.091974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091908    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2shb6\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-kube-api-access-2shb6\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.091974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091931    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-web-config\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.091974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091956    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-1\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.091979    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-2\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092001    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-0\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092032    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-kube-rbac-proxy\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092074    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-thanos-prometheus-http-client-file\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092106    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-trusted-ca-bundle\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092129    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092177    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-metrics-client-ca\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092207    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-metrics-client-certs\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092232    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092243    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092293    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-config\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092355    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-tls-assets\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092383    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-config-out\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.092559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092418    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-tls\") pod \"18fabdbc-0648-42f7-8acc-30187ae19e69\" (UID: \"18fabdbc-0648-42f7-8acc-30187ae19e69\") "
Sep 09 20:14:13.093928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092644    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.093928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092716    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.093928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092735    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.093928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092785    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.093928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.092964    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.095397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.094332    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.095397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.094614    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.096462 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.096436    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:14:13.096574 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.096496    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-config" (OuterVolumeSpecName: "config") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.096694 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.096451    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.096908 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.096868    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-kube-api-access-2shb6" (OuterVolumeSpecName: "kube-api-access-2shb6") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "kube-api-access-2shb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:14:13.098270 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.098220    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-config-out" (OuterVolumeSpecName: "config-out") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:14:13.098404 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.098332    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.098720 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.098696    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.101058 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.101020    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:14:13.101298 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.101272    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.101504 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.101484    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.101589 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.101563    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.101786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.101754    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.101870 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.101850    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:14:13.109747 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.109718    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-web-config" (OuterVolumeSpecName: "web-config") pod "18fabdbc-0648-42f7-8acc-30187ae19e69" (UID: "18fabdbc-0648-42f7-8acc-30187ae19e69"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193085    2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-kube-rbac-proxy\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193115    2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193126    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193135    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193145    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-metrics-client-ca\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193154    2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-metrics-client-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193162    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193171    2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-config\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193179    2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-tls-assets\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193187    2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-config-out\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193194    2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193202    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-db\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193210    2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-secret-grpc-tls\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193218    2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193227    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2shb6\" (UniqueName: \"kubernetes.io/projected/18fabdbc-0648-42f7-8acc-30187ae19e69-kube-api-access-2shb6\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193236    2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/18fabdbc-0648-42f7-8acc-30187ae19e69-web-config\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193244    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.193527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.193276    2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/18fabdbc-0648-42f7-8acc-30187ae19e69-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:14:13.570176 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.570089    2573 generic.go:356] "Generic (PLEG): container finished" podID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerID="32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721" exitCode=0
Sep 09 20:14:13.570176 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.570167    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.570363 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.570165    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721"}
Sep 09 20:14:13.570363 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.570333    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"18fabdbc-0648-42f7-8acc-30187ae19e69","Type":"ContainerDied","Data":"1ad293fa691ce7d8c0fa1a737678564945db8e71000f91e7c831557825f105b9"}
Sep 09 20:14:13.570431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.570366    2573 scope.go:117] "RemoveContainer" containerID="edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb"
Sep 09 20:14:13.578542 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.578525    2573 scope.go:117] "RemoveContainer" containerID="e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4"
Sep 09 20:14:13.585149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.585132    2573 scope.go:117] "RemoveContainer" containerID="32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721"
Sep 09 20:14:13.591841 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.591825    2573 scope.go:117] "RemoveContainer" containerID="55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2"
Sep 09 20:14:13.595112 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.595084    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:14:13.599384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.599363    2573 scope.go:117] "RemoveContainer" containerID="484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe"
Sep 09 20:14:13.601360 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.601340    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:14:13.606111 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.606094    2573 scope.go:117] "RemoveContainer" containerID="d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9"
Sep 09 20:14:13.613009 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.612993    2573 scope.go:117] "RemoveContainer" containerID="9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040"
Sep 09 20:14:13.619452 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.619388    2573 scope.go:117] "RemoveContainer" containerID="edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb"
Sep 09 20:14:13.619705 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.619686    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb\": container with ID starting with edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb not found: ID does not exist" containerID="edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb"
Sep 09 20:14:13.619762 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.619712    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb"} err="failed to get container status \"edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb\": rpc error: code = NotFound desc = could not find container \"edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb\": container with ID starting with edb2651da3a35a38a3a50d8fe7479bddc35fd65fdc353487e2074065a1e2e1bb not found: ID does not exist"
Sep 09 20:14:13.619762 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.619730    2573 scope.go:117] "RemoveContainer" containerID="e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4"
Sep 09 20:14:13.619990 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.619970    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4\": container with ID starting with e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4 not found: ID does not exist" containerID="e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4"
Sep 09 20:14:13.620031 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.619997    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4"} err="failed to get container status \"e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4\": rpc error: code = NotFound desc = could not find container \"e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4\": container with ID starting with e5274ba4ed9e122a76c59b7636fc32d8823fbbe50b518a9ad177db11c7e6a6e4 not found: ID does not exist"
Sep 09 20:14:13.620031 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620014    2573 scope.go:117] "RemoveContainer" containerID="32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721"
Sep 09 20:14:13.620283 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.620244    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721\": container with ID starting with 32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721 not found: ID does not exist" containerID="32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721"
Sep 09 20:14:13.620386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620286    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721"} err="failed to get container status \"32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721\": rpc error: code = NotFound desc = could not find container \"32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721\": container with ID starting with 32a5fe0d1a4dd6911d8446c8d58119d2b356796ce8a9ef2f2100c867feccb721 not found: ID does not exist"
Sep 09 20:14:13.620386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620309    2573 scope.go:117] "RemoveContainer" containerID="55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2"
Sep 09 20:14:13.620565 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.620548    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2\": container with ID starting with 55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2 not found: ID does not exist" containerID="55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2"
Sep 09 20:14:13.620640 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620571    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2"} err="failed to get container status \"55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2\": rpc error: code = NotFound desc = could not find container \"55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2\": container with ID starting with 55fea23f9e4c102782496af3e84b0e29fcd0cf2bba572424d614cd05dfdd16b2 not found: ID does not exist"
Sep 09 20:14:13.620640 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620586    2573 scope.go:117] "RemoveContainer" containerID="484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe"
Sep 09 20:14:13.620842 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.620823    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe\": container with ID starting with 484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe not found: ID does not exist" containerID="484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe"
Sep 09 20:14:13.620882 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620847    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe"} err="failed to get container status \"484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe\": rpc error: code = NotFound desc = could not find container \"484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe\": container with ID starting with 484ff6ba2289305e6c5f13e857b49c82391c309625b7f8de3a2729dcf84612fe not found: ID does not exist"
Sep 09 20:14:13.620882 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.620862    2573 scope.go:117] "RemoveContainer" containerID="d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9"
Sep 09 20:14:13.621114 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.621098    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9\": container with ID starting with d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9 not found: ID does not exist" containerID="d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9"
Sep 09 20:14:13.621168 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.621118    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9"} err="failed to get container status \"d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9\": rpc error: code = NotFound desc = could not find container \"d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9\": container with ID starting with d4effc7e7785ea78f22279e7b12e775ddc25c1e6dd3d2f2e2886fd1fb137a4d9 not found: ID does not exist"
Sep 09 20:14:13.621168 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.621136    2573 scope.go:117] "RemoveContainer" containerID="9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040"
Sep 09 20:14:13.621377 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:14:13.621361    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040\": container with ID starting with 9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040 not found: ID does not exist" containerID="9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040"
Sep 09 20:14:13.621420 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.621380    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040"} err="failed to get container status \"9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040\": rpc error: code = NotFound desc = could not find container \"9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040\": container with ID starting with 9a404f47af1963dd6f64d4e80fd567d6521301508dbc5692da8e4fc5105fd040 not found: ID does not exist"
Sep 09 20:14:13.628331 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628299    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:14:13.628569 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628558    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="config-reloader"
Sep 09 20:14:13.628605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628571    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="config-reloader"
Sep 09 20:14:13.628605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628579    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-thanos"
Sep 09 20:14:13.628605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628585    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-thanos"
Sep 09 20:14:13.628605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628594    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy"
Sep 09 20:14:13.628605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628599    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628607    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="prometheus"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628612    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="prometheus"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628619    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-web"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628624    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-web"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628633    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c72e7028-68f7-4111-8461-15906e90decc" containerName="registry"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628637    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c72e7028-68f7-4111-8461-15906e90decc" containerName="registry"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628643    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="init-config-reloader"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628649    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="init-config-reloader"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628662    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="thanos-sidecar"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628667    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="thanos-sidecar"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628726    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-web"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628732    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="prometheus"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628740    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy-thanos"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628745    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c72e7028-68f7-4111-8461-15906e90decc" containerName="registry"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628751    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="config-reloader"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628756    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="thanos-sidecar"
Sep 09 20:14:13.628786 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.628762    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" containerName="kube-rbac-proxy"
Sep 09 20:14:13.633761 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.633739    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.636515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636494    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 20:14:13.636604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636555    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 20:14:13.636766 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636747    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 20:14:13.636897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636809    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 20:14:13.636897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636841    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qq89s\""
Sep 09 20:14:13.636897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636850    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 20:14:13.636897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.636856    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 20:14:13.637183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.637000    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 20:14:13.637183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.637132    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 20:14:13.637355 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.637185    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 20:14:13.637355 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.637200    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cto4gt88risbs\""
Sep 09 20:14:13.637510 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.637407    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 20:14:13.637740 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.637719    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 20:14:13.638082 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.638065    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 20:14:13.640061 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.640042    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 20:14:13.642846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.642829    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 20:14:13.643220 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.643201    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18fabdbc-0648-42f7-8acc-30187ae19e69" path="/var/lib/kubelet/pods/18fabdbc-0648-42f7-8acc-30187ae19e69/volumes"
Sep 09 20:14:13.650323 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.650301    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:14:13.697149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697146    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2279281c-31c4-460d-80da-0f9a4cae52ce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697164    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697189    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697230    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-web-config\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697289    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-config\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697308    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697330    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697350    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2279281c-31c4-460d-80da-0f9a4cae52ce-config-out\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697369    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697386    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfv8d\" (UniqueName: \"kubernetes.io/projected/2279281c-31c4-460d-80da-0f9a4cae52ce-kube-api-access-xfv8d\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697443    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697458    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697530    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697563    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697597    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697635    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697664    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.697826 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.697688    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799000    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799034    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2279281c-31c4-460d-80da-0f9a4cae52ce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799098    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799130    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-web-config\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799158    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-config\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799183    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799226    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799278    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2279281c-31c4-460d-80da-0f9a4cae52ce-config-out\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799301    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799319    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799347    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfv8d\" (UniqueName: \"kubernetes.io/projected/2279281c-31c4-460d-80da-0f9a4cae52ce-kube-api-access-xfv8d\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799375    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799393    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799417    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799436    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799494    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799521    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.799692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799548    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.800149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799943    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.800149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.799972    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.800280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.800240    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.801285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.801240    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.802077 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.802403 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802375    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-config\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.802489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802411    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2279281c-31c4-460d-80da-0f9a4cae52ce-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.802489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2279281c-31c4-460d-80da-0f9a4cae52ce-config-out\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.802489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.802942 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802919    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.803029 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802980    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.803029 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.802996    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-web-config\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.803622 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.803595    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.803745 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.803723    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2279281c-31c4-460d-80da-0f9a4cae52ce-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.804575 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.804554    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.804690 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.804671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.804934 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.804915    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.805361 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.805337    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.805437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.805401    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2279281c-31c4-460d-80da-0f9a4cae52ce-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.816214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.816188    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfv8d\" (UniqueName: \"kubernetes.io/projected/2279281c-31c4-460d-80da-0f9a4cae52ce-kube-api-access-xfv8d\") pod \"prometheus-k8s-0\" (UID: \"2279281c-31c4-460d-80da-0f9a4cae52ce\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:13.944707 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:13.944680    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:14.241463 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.241438    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:14:14.243186 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:14:14.243160    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2279281c_31c4_460d_80da_0f9a4cae52ce.slice/crio-e912902b9d12388fd6e1f6973aa4cf07921700bf9a5c4d119428b807dd7fa668 WatchSource:0}: Error finding container e912902b9d12388fd6e1f6973aa4cf07921700bf9a5c4d119428b807dd7fa668: Status 404 returned error can't find the container with id e912902b9d12388fd6e1f6973aa4cf07921700bf9a5c4d119428b807dd7fa668
Sep 09 20:14:14.531403 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.531315    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:14.582208 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.582165    2573 generic.go:356] "Generic (PLEG): container finished" podID="2279281c-31c4-460d-80da-0f9a4cae52ce" containerID="7596e86c9575efbf0acaf35eb155c758814e4d4a2299af9b9187bb11ceac4f83" exitCode=0
Sep 09 20:14:14.582374 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.582263    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerDied","Data":"7596e86c9575efbf0acaf35eb155c758814e4d4a2299af9b9187bb11ceac4f83"}
Sep 09 20:14:14.582433 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.582394    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"e912902b9d12388fd6e1f6973aa4cf07921700bf9a5c4d119428b807dd7fa668"}
Sep 09 20:14:14.634950 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.634828    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:14.693604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:14.693575    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:15.261285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.261230    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:15.333461 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.333429    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:15.590483 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590409    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:14:15.590796 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590776    2573 generic.go:356] "Generic (PLEG): container finished" podID="2279281c-31c4-460d-80da-0f9a4cae52ce" containerID="68b7f07c2df8d548aa6456d59a863c5356ae466f1fed1077f3c550b79bc6386b" exitCode=2
Sep 09 20:14:15.590869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590830    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"09825b1a25c930cf3351b34b65412ab490652455f96195b283327ed47c7200b8"}
Sep 09 20:14:15.590908 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590890    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"f13630faae5e50d6b605ce7029a7bb1d5eb7a51403e3ed0e4316f4b830b4eb75"}
Sep 09 20:14:15.590938 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590919    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"31c65d4fad81da1bacaff6c17f763349282d5790c41cc6ae50e57ba27ed66fda"}
Sep 09 20:14:15.590967 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590954    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"f988afddeeeb23e8ede73ca40b64d12fec85c1c78673b3e40ca143055efe58a1"}
Sep 09 20:14:15.590999 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.590982    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"e3d78ada6707bd0ee93f0784249ac0225d4bb5a6b06fe657ac15478955546261"}
Sep 09 20:14:15.591040 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.591009    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerDied","Data":"68b7f07c2df8d548aa6456d59a863c5356ae466f1fed1077f3c550b79bc6386b"}
Sep 09 20:14:15.591353 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:15.591340    2573 scope.go:117] "RemoveContainer" containerID="68b7f07c2df8d548aa6456d59a863c5356ae466f1fed1077f3c550b79bc6386b"
Sep 09 20:14:16.597703 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:16.597678    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:14:16.598116 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:16.598071    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2279281c-31c4-460d-80da-0f9a4cae52ce","Type":"ContainerStarted","Data":"5727fc9ffc29a2ce96478fc63793567f3b65450e415a2fc30d5805f24bf85b2f"}
Sep 09 20:14:16.632342 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:16.632287    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.615383842 podStartE2EDuration="3.632270186s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.886344ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:14:16.629106978 +0000 UTC m=+195.560190297" watchObservedRunningTime="2026-09-09 20:14:16.632270186 +0000 UTC m=+195.563353503"
Sep 09 20:14:18.945161 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:18.945112    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:14:25.678057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:25.678020    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:25.732372 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:25.732342    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:25.795311 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:25.795281    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:14:42.987071 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:14:42.987040    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:06.634473 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:06.634436    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:13.945400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:13.945366    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:15:13.959622 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:13.959600    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:15:14.834966 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:14.834933    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:15:21.633628 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:21.633586    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:21.703431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:21.703393    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:24.888319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:24.888217    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:33.513535 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:33.513494    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:33.571426 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:33.571403    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:35.925436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:35.925403    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:36.025595 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:36.025567    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:36.098259 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:36.098236    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:50.575318 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:50.575280    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:15:50.676144 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:15:50.676110    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:16:01.436800 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:01.436767    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:16:01.441128 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:01.441102    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:16:01.449118 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:01.449092    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:16:01.453978 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:01.453960    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:16:01.455285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:01.455267    2573 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 20:16:15.568665 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:15.568619    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:16:48.529684 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:16:48.529648    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:00.299968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:00.299862    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:00.399414 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:00.399356    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:00.459835 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:00.459805    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:27.455703 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:27.455667    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:27.549545 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:27.549493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:32.917116 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:32.917081    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:33.205198 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:33.205123    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:33.398168 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:33.398130    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:33.588706 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:33.588628    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:33.821262 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:33.821231    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:33.922199 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:33.922169    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:34.026149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:34.026114    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:34.099263 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:34.099203    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:34.295609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:34.295529    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:34.396552 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:34.396477    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:34.517417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:34.517386    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:17:34.592954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:17:34.592882    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:01.311656 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:01.311605    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:01.379174 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:01.379145    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:12.501558 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:12.501526    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:12.605888 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:12.605861    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:12.707472 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:12.707445    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:26.342183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:26.342110    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:26.442352 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:26.442316    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:30.000298 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:30.000268    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:30.061196 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:30.061162    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:31.958902 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:31.958871    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:32.017672 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:32.017641    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:42.548901 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:42.548860    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:43.416268 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:43.416205    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:46.168355 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:46.168315    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:46.268976 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:46.268936    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:48.491586 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:48.491553    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:48.594583 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:48.594551    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:49.502214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.502178    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7nnb8"]
Sep 09 20:18:49.505554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.505530    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.507906 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.507883    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zmbsl\""
Sep 09 20:18:49.508068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.508049    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 20:18:49.508932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.508917    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 20:18:49.517445 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.517425    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7nnb8"]
Sep 09 20:18:49.627546 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.627514    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/790e7c46-dc52-419d-8265-c586fd8beaf8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7nnb8\" (UID: \"790e7c46-dc52-419d-8265-c586fd8beaf8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.627723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.627621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76ntl\" (UniqueName: \"kubernetes.io/projected/790e7c46-dc52-419d-8265-c586fd8beaf8-kube-api-access-76ntl\") pod \"cert-manager-webhook-6486ffd997-7nnb8\" (UID: \"790e7c46-dc52-419d-8265-c586fd8beaf8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.728497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.728465    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76ntl\" (UniqueName: \"kubernetes.io/projected/790e7c46-dc52-419d-8265-c586fd8beaf8-kube-api-access-76ntl\") pod \"cert-manager-webhook-6486ffd997-7nnb8\" (UID: \"790e7c46-dc52-419d-8265-c586fd8beaf8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.728658 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.728509    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/790e7c46-dc52-419d-8265-c586fd8beaf8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7nnb8\" (UID: \"790e7c46-dc52-419d-8265-c586fd8beaf8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.737674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.737639    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/790e7c46-dc52-419d-8265-c586fd8beaf8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7nnb8\" (UID: \"790e7c46-dc52-419d-8265-c586fd8beaf8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.737880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.737857    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76ntl\" (UniqueName: \"kubernetes.io/projected/790e7c46-dc52-419d-8265-c586fd8beaf8-kube-api-access-76ntl\") pod \"cert-manager-webhook-6486ffd997-7nnb8\" (UID: \"790e7c46-dc52-419d-8265-c586fd8beaf8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:49.824918 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:49.824845    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:50.097562 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.097483    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7nnb8"]
Sep 09 20:18:50.100262 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:18:50.100216    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod790e7c46_dc52_419d_8265_c586fd8beaf8.slice/crio-b88ff118d5438dce9ab166a0ad56a34c74b4474ba62166e1a53cf2c45934e599 WatchSource:0}: Error finding container b88ff118d5438dce9ab166a0ad56a34c74b4474ba62166e1a53cf2c45934e599: Status 404 returned error can't find the container with id b88ff118d5438dce9ab166a0ad56a34c74b4474ba62166e1a53cf2c45934e599
Sep 09 20:18:50.102413 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.102398    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:18:50.634942 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.634907    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8" event={"ID":"790e7c46-dc52-419d-8265-c586fd8beaf8","Type":"ContainerStarted","Data":"b88ff118d5438dce9ab166a0ad56a34c74b4474ba62166e1a53cf2c45934e599"}
Sep 09 20:18:50.726213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.726183    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:50.789337 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.789298    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"]
Sep 09 20:18:50.793541 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.793522    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:50.795720 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.795700    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-s9gzc\""
Sep 09 20:18:50.802370 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.802347    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"]
Sep 09 20:18:50.851682 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.851640    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:50.937545 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.937463    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9361767-6b50-4010-b350-aabf4ab32476-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ncvrr\" (UID: \"c9361767-6b50-4010-b350-aabf4ab32476\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:50.937545 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.937518    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksb2g\" (UniqueName: \"kubernetes.io/projected/c9361767-6b50-4010-b350-aabf4ab32476-kube-api-access-ksb2g\") pod \"cert-manager-cainjector-559cdd8c87-ncvrr\" (UID: \"c9361767-6b50-4010-b350-aabf4ab32476\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:50.949946 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:50.949918    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:51.038351 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.038317    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksb2g\" (UniqueName: \"kubernetes.io/projected/c9361767-6b50-4010-b350-aabf4ab32476-kube-api-access-ksb2g\") pod \"cert-manager-cainjector-559cdd8c87-ncvrr\" (UID: \"c9361767-6b50-4010-b350-aabf4ab32476\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:51.038547 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.038407    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9361767-6b50-4010-b350-aabf4ab32476-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ncvrr\" (UID: \"c9361767-6b50-4010-b350-aabf4ab32476\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:51.047457 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.047425    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9361767-6b50-4010-b350-aabf4ab32476-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ncvrr\" (UID: \"c9361767-6b50-4010-b350-aabf4ab32476\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:51.047572 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.047462    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksb2g\" (UniqueName: \"kubernetes.io/projected/c9361767-6b50-4010-b350-aabf4ab32476-kube-api-access-ksb2g\") pod \"cert-manager-cainjector-559cdd8c87-ncvrr\" (UID: \"c9361767-6b50-4010-b350-aabf4ab32476\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:51.102478 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.102444    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"
Sep 09 20:18:51.379344 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.379280    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr"]
Sep 09 20:18:51.383790 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:18:51.383739    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9361767_6b50_4010_b350_aabf4ab32476.slice/crio-b26e45c763fe06ddd796c46e549be30bb8af67383662a73afc47d07ed5b694af WatchSource:0}: Error finding container b26e45c763fe06ddd796c46e549be30bb8af67383662a73afc47d07ed5b694af: Status 404 returned error can't find the container with id b26e45c763fe06ddd796c46e549be30bb8af67383662a73afc47d07ed5b694af
Sep 09 20:18:51.643899 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:51.643825    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr" event={"ID":"c9361767-6b50-4010-b350-aabf4ab32476","Type":"ContainerStarted","Data":"b26e45c763fe06ddd796c46e549be30bb8af67383662a73afc47d07ed5b694af"}
Sep 09 20:18:52.919638 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:52.919585    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:53.022438 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:53.022401    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:53.122841 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:53.122807    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:18:54.657873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:54.657839    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr" event={"ID":"c9361767-6b50-4010-b350-aabf4ab32476","Type":"ContainerStarted","Data":"80079f845f782704291d1e6306509530b096144550fd046a3bdd1114df9812ae"}
Sep 09 20:18:54.659500 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:54.659475    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8" event={"ID":"790e7c46-dc52-419d-8265-c586fd8beaf8","Type":"ContainerStarted","Data":"fc65f7cc9356237b613649770e7cbc1865f4250a3be5272cb332baac41eea139"}
Sep 09 20:18:54.659605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:54.659529    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:18:54.672988 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:54.672947    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ncvrr" podStartSLOduration=2.141701337 podStartE2EDuration="4.672929593s" totalImagesPullingTime="2.531228256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:18:54.672378733 +0000 UTC m=+473.603462053" watchObservedRunningTime="2026-09-09 20:18:54.672929593 +0000 UTC m=+473.604012915"
Sep 09 20:18:54.701124 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:54.701069    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8" podStartSLOduration=1.898179255 podStartE2EDuration="5.701053816s" totalImagesPullingTime="3.802874561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:18:54.699889863 +0000 UTC m=+473.630973182" watchObservedRunningTime="2026-09-09 20:18:54.701053816 +0000 UTC m=+473.632137135"
Sep 09 20:18:59.121918 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:18:59.121882    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:00.665025 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:00.664993    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7nnb8"
Sep 09 20:19:09.217615 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.217575    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z2p7f"]
Sep 09 20:19:09.220929 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.220905    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.225110 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.225088    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7gzv6\""
Sep 09 20:19:09.229438 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.229415    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z2p7f"]
Sep 09 20:19:09.290540 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.290504    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd02f51f-126d-482a-b5d9-056a9bb506bb-bound-sa-token\") pod \"cert-manager-55b4d954d5-z2p7f\" (UID: \"fd02f51f-126d-482a-b5d9-056a9bb506bb\") " pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.290719 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.290659    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgf84\" (UniqueName: \"kubernetes.io/projected/fd02f51f-126d-482a-b5d9-056a9bb506bb-kube-api-access-wgf84\") pod \"cert-manager-55b4d954d5-z2p7f\" (UID: \"fd02f51f-126d-482a-b5d9-056a9bb506bb\") " pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.391652 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.391618    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd02f51f-126d-482a-b5d9-056a9bb506bb-bound-sa-token\") pod \"cert-manager-55b4d954d5-z2p7f\" (UID: \"fd02f51f-126d-482a-b5d9-056a9bb506bb\") " pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.391835 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.391669    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgf84\" (UniqueName: \"kubernetes.io/projected/fd02f51f-126d-482a-b5d9-056a9bb506bb-kube-api-access-wgf84\") pod \"cert-manager-55b4d954d5-z2p7f\" (UID: \"fd02f51f-126d-482a-b5d9-056a9bb506bb\") " pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.400915 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.400888    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd02f51f-126d-482a-b5d9-056a9bb506bb-bound-sa-token\") pod \"cert-manager-55b4d954d5-z2p7f\" (UID: \"fd02f51f-126d-482a-b5d9-056a9bb506bb\") " pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.401036 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.400958    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgf84\" (UniqueName: \"kubernetes.io/projected/fd02f51f-126d-482a-b5d9-056a9bb506bb-kube-api-access-wgf84\") pod \"cert-manager-55b4d954d5-z2p7f\" (UID: \"fd02f51f-126d-482a-b5d9-056a9bb506bb\") " pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.531102 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.530984    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z2p7f"
Sep 09 20:19:09.801804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:09.801774    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z2p7f"]
Sep 09 20:19:09.804356 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:19:09.804322    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd02f51f_126d_482a_b5d9_056a9bb506bb.slice/crio-a11920da2b60c095947d73615a019914e13bd7d25ce10163d72d2d3d83023338 WatchSource:0}: Error finding container a11920da2b60c095947d73615a019914e13bd7d25ce10163d72d2d3d83023338: Status 404 returned error can't find the container with id a11920da2b60c095947d73615a019914e13bd7d25ce10163d72d2d3d83023338
Sep 09 20:19:10.725808 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:10.725768    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z2p7f" event={"ID":"fd02f51f-126d-482a-b5d9-056a9bb506bb","Type":"ContainerStarted","Data":"0c56e4c01d5a86df71fea4766eba9456a08941117a67e748e236f10699973b5f"}
Sep 09 20:19:10.726194 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:10.725821    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z2p7f" event={"ID":"fd02f51f-126d-482a-b5d9-056a9bb506bb","Type":"ContainerStarted","Data":"a11920da2b60c095947d73615a019914e13bd7d25ce10163d72d2d3d83023338"}
Sep 09 20:19:10.744110 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:10.744065    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z2p7f" podStartSLOduration=1.744048923 podStartE2EDuration="1.744048923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:19:10.74246261 +0000 UTC m=+489.673545929" watchObservedRunningTime="2026-09-09 20:19:10.744048923 +0000 UTC m=+489.675132242"
Sep 09 20:19:10.762239 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:10.762209    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:10.858157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:10.858128    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:12.643197 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:12.643163    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:12.741029 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:12.740993    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:15.092888 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:15.092852    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:21.770805 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:21.770774    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:26.268080 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:26.268044    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:26.328985 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:26.328952    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:27.320105 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:27.320071    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:30.296317 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:30.296282    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:30.396981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:30.396946    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:31.989469 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:31.989443    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:32.030686 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.030656    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"]
Sep 09 20:19:32.035278 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.035236    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.037676 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.037657    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 20:19:32.037789 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.037699    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 20:19:32.038669 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.038650    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 20:19:32.038772 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.038668    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 20:19:32.038772 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.038684    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:19:32.038772 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.038697    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h728d\""
Sep 09 20:19:32.045496 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.045473    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"]
Sep 09 20:19:32.068567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.068535    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvpk\" (UniqueName: \"kubernetes.io/projected/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-kube-api-access-zjvpk\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.068723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.068574    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-metrics-cert\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.068723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.068598    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-manager-config\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.068723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.068617    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-cert\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.075747 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.075724    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:32.169222 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.169188    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvpk\" (UniqueName: \"kubernetes.io/projected/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-kube-api-access-zjvpk\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.169409 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.169231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-metrics-cert\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.169409 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.169266    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-manager-config\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.169533 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.169451    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-cert\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.170084 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.170050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-manager-config\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.171674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.171645    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-metrics-cert\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.171853 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.171832    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-cert\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.182711 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.182686    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvpk\" (UniqueName: \"kubernetes.io/projected/d6d40538-f779-4dad-b8a9-3b0f50f6fe31-kube-api-access-zjvpk\") pod \"lws-controller-manager-7f7677c75-8nq6x\" (UID: \"d6d40538-f779-4dad-b8a9-3b0f50f6fe31\") " pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.345489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.345396    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:32.616924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.616900    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"]
Sep 09 20:19:32.618887 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:19:32.618860    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d40538_f779_4dad_b8a9_3b0f50f6fe31.slice/crio-a95c0865946272c8a5a4f1cf577cde55ec472341242b10d2abf062ab18dd6485 WatchSource:0}: Error finding container a95c0865946272c8a5a4f1cf577cde55ec472341242b10d2abf062ab18dd6485: Status 404 returned error can't find the container with id a95c0865946272c8a5a4f1cf577cde55ec472341242b10d2abf062ab18dd6485
Sep 09 20:19:32.819964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:32.819924    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x" event={"ID":"d6d40538-f779-4dad-b8a9-3b0f50f6fe31","Type":"ContainerStarted","Data":"a95c0865946272c8a5a4f1cf577cde55ec472341242b10d2abf062ab18dd6485"}
Sep 09 20:19:33.072567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:33.072531    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:33.140911 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:33.140869    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:35.835343 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:35.835309    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x" event={"ID":"d6d40538-f779-4dad-b8a9-3b0f50f6fe31","Type":"ContainerStarted","Data":"84e9fc2617ffea18027563f463951bfda751fdea4def1dbc47bf528d8e26d40a"}
Sep 09 20:19:35.835695 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:35.835443    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:35.852232 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:35.852187    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x" podStartSLOduration=1.123633574 podStartE2EDuration="3.852173738s" totalImagesPullingTime="2.728540164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:19:35.850980212 +0000 UTC m=+514.782063531" watchObservedRunningTime="2026-09-09 20:19:35.852173738 +0000 UTC m=+514.783257055"
Sep 09 20:19:35.942585 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:35.942556    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:38.537879 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:38.537842    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:38.642601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:38.642571    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:43.961565 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:43.961527    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:44.044951 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:44.044921    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:19:46.841371 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:46.841342    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f7677c75-8nq6x"
Sep 09 20:19:57.664724 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:19:57.664639    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:00.450928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:00.450893    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:02.154870 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:02.154840    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:02.258694 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:02.258659    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:02.868703 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:02.868669    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:03.251512 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:03.251418    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:03.493566 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:03.493533    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:04.505208 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:04.505161    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:04.615725 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:04.615698    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:06.451340 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:06.451297    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:06.647875 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:06.647826    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:08.500450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.500403    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:08.579601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.579569    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"]
Sep 09 20:20:08.587555 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.587534    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:08.590683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.590664    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 20:20:08.591542 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.591523    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 20:20:08.592071 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.592057    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-54rj6\""
Sep 09 20:20:08.602055 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.602034    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"]
Sep 09 20:20:08.604283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.604246    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:08.773430 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.773339    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb6bm\" (UniqueName: \"kubernetes.io/projected/805aa70d-a43f-47c7-a9be-58372b88cc06-kube-api-access-rb6bm\") pod \"authorino-operator-6fc8f864f4-mjx5g\" (UID: \"805aa70d-a43f-47c7-a9be-58372b88cc06\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:08.874475 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.874440    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb6bm\" (UniqueName: \"kubernetes.io/projected/805aa70d-a43f-47c7-a9be-58372b88cc06-kube-api-access-rb6bm\") pod \"authorino-operator-6fc8f864f4-mjx5g\" (UID: \"805aa70d-a43f-47c7-a9be-58372b88cc06\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:08.884820 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.884789    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb6bm\" (UniqueName: \"kubernetes.io/projected/805aa70d-a43f-47c7-a9be-58372b88cc06-kube-api-access-rb6bm\") pod \"authorino-operator-6fc8f864f4-mjx5g\" (UID: \"805aa70d-a43f-47c7-a9be-58372b88cc06\") " pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:08.897603 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:08.897584    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:09.171141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:09.170911    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"]
Sep 09 20:20:09.173883 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:20:09.173859    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805aa70d_a43f_47c7_a9be_58372b88cc06.slice/crio-9c827af248b45107b59ecde4eff4b29c5f3c05425923756c5a50ab82135d7613 WatchSource:0}: Error finding container 9c827af248b45107b59ecde4eff4b29c5f3c05425923756c5a50ab82135d7613: Status 404 returned error can't find the container with id 9c827af248b45107b59ecde4eff4b29c5f3c05425923756c5a50ab82135d7613
Sep 09 20:20:09.973329 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:09.973294    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g" event={"ID":"805aa70d-a43f-47c7-a9be-58372b88cc06","Type":"ContainerStarted","Data":"9c827af248b45107b59ecde4eff4b29c5f3c05425923756c5a50ab82135d7613"}
Sep 09 20:20:10.071910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:10.071879    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:10.173756 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:10.173714    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:12.915913 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:12.915871    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:12.990632 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:12.990596    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g" event={"ID":"805aa70d-a43f-47c7-a9be-58372b88cc06","Type":"ContainerStarted","Data":"a7c8057d76129fe18d2137da6e5f56dfba26ae198379ed3806c6a014e360cab2"}
Sep 09 20:20:12.990817 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:12.990686    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:13.013489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:13.013438    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g" podStartSLOduration=2.014964302 podStartE2EDuration="5.013422937s" totalImagesPullingTime="2.998458635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:20:13.012034153 +0000 UTC m=+551.943117495" watchObservedRunningTime="2026-09-09 20:20:13.013422937 +0000 UTC m=+551.944506256"
Sep 09 20:20:13.620907 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:13.620875    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:13.859107 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:13.859071    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:13.958804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:13.958728    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:17.678778 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:17.678746    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:17.780916 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:17.780887    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:17.882772 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:17.882732    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:17.980875 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:17.980814    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:23.997405 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:23.997371    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6fc8f864f4-mjx5g"
Sep 09 20:20:28.592139 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:28.592109    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:28.694854 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:28.694822    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:29.209981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:29.209951    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:29.306323 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:29.306290    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:34.380049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:34.380018    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:34.480965 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:34.480937    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:37.799523 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:37.799483    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:37.896449 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:37.896396    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:42.383436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:42.383389    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:42.528866 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:42.528831    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:42.775824 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:42.775711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:42.876490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:42.876462    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:43.185144 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:43.185111    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:43.251776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:43.251716    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:44.589970 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:44.589937    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:48.229010 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:48.228975    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:48.329428 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:48.329399    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:51.191111 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:51.191074    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:51.293864 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:51.293833    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:56.146292 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.146245    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:56.331699 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.331670    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:56.432237 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.432127    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:56.549761 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.549726    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"]
Sep 09 20:20:56.552864 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.552848    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.555277 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.555241    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 20:20:56.555416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.555324    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9mpkd\""
Sep 09 20:20:56.565805 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.565783    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"]
Sep 09 20:20:56.593092 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.593058    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"]
Sep 09 20:20:56.668342 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.668306    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hczt\" (UniqueName: \"kubernetes.io/projected/8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3-kube-api-access-7hczt\") pod \"limitador-limitador-84994f7ffc-2q7lb\" (UID: \"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.668491 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.668365    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3-config-file\") pod \"limitador-limitador-84994f7ffc-2q7lb\" (UID: \"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.769206 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.769127    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hczt\" (UniqueName: \"kubernetes.io/projected/8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3-kube-api-access-7hczt\") pod \"limitador-limitador-84994f7ffc-2q7lb\" (UID: \"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.769206 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.769179    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3-config-file\") pod \"limitador-limitador-84994f7ffc-2q7lb\" (UID: \"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.769770 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.769751    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3-config-file\") pod \"limitador-limitador-84994f7ffc-2q7lb\" (UID: \"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.777794 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.777770    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hczt\" (UniqueName: \"kubernetes.io/projected/8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3-kube-api-access-7hczt\") pod \"limitador-limitador-84994f7ffc-2q7lb\" (UID: \"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:56.862655 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:56.862619    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:57.034061 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:57.033962    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:57.133199 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:57.133167    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"]
Sep 09 20:20:57.133294 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:57.133157    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:20:57.135452 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:20:57.135426    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f41ad76_b2f3_4b74_b6a8_18c06fc4a8a3.slice/crio-5ec17b35c87ed66e519e8383e867b48370556b63f09e90d50911cc451b7c75bd WatchSource:0}: Error finding container 5ec17b35c87ed66e519e8383e867b48370556b63f09e90d50911cc451b7c75bd: Status 404 returned error can't find the container with id 5ec17b35c87ed66e519e8383e867b48370556b63f09e90d50911cc451b7c75bd
Sep 09 20:20:57.184105 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:57.184071    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb" event={"ID":"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3","Type":"ContainerStarted","Data":"5ec17b35c87ed66e519e8383e867b48370556b63f09e90d50911cc451b7c75bd"}
Sep 09 20:20:59.194520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:59.194488    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb" event={"ID":"8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3","Type":"ContainerStarted","Data":"da9f409ea0b985fc9280c762e45bbb38daf441ddabff346cafa6116077bea5fb"}
Sep 09 20:20:59.194940 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:59.194563    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:20:59.212503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:20:59.212458    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb" podStartSLOduration=2.067549642 podStartE2EDuration="3.212441027s" totalImagesPullingTime="1.144891385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:20:59.210520221 +0000 UTC m=+598.141603541" watchObservedRunningTime="2026-09-09 20:20:59.212441027 +0000 UTC m=+598.143524373"
Sep 09 20:21:01.467772 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:01.467742    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:21:01.468199 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:01.468180    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:21:01.479267 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:01.479235    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:21:01.480004 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:01.479977    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:21:06.391780 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:06.391734    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:21:10.200275 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:10.200229    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-2q7lb"
Sep 09 20:21:16.255695 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:16.255656    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:21:16.407424 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:16.407392    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:21:24.877369 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:24.877280    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:21:24.978199 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:24.978168    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:21:50.522204 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:50.522167    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:21:51.956754 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:21:51.956719    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:22:23.890746 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:22:23.890711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:22:28.269997 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:22:28.269951    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:22:28.326733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:22:28.326705    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:22:59.930996 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:22:59.930899    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:05.879999 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:05.879961    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:05.944222 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:05.944170    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:06.073637 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:06.073606    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:06.174278 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:06.174174    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:06.335962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:06.335931    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:06.882284 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:06.882231    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:23:06.943445 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:23:06.943399    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:24:39.400426 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:24:39.400342    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:24:39.487196 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:24:39.487147    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:24:51.883930 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:24:51.883892    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:24:51.978213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:24:51.978185    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:25:00.890290 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:00.890218    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:25:20.377790 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:20.377751    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:25:25.594764 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.594733    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:25:25.902008 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.901973    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"]
Sep 09 20:25:25.905734 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.905710    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:25.909350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.909065    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-f6phs\""
Sep 09 20:25:25.909350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.909128    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Sep 09 20:25:25.909350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.909073    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Sep 09 20:25:25.909350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.909073    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Sep 09 20:25:25.909350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.909073    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-q8c6c\""
Sep 09 20:25:25.913897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.913868    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"]
Sep 09 20:25:25.941469 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:25.941442    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:25:26.025150 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.025112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-575vx\" (UniqueName: \"kubernetes.io/projected/2a94a786-51d8-4769-b320-40b215ac1147-kube-api-access-575vx\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.025358 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.025245    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a94a786-51d8-4769-b320-40b215ac1147-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.126668 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.126621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-575vx\" (UniqueName: \"kubernetes.io/projected/2a94a786-51d8-4769-b320-40b215ac1147-kube-api-access-575vx\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.126837 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.126725    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a94a786-51d8-4769-b320-40b215ac1147-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.129169 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.129139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a94a786-51d8-4769-b320-40b215ac1147-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.134596 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.134573    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-575vx\" (UniqueName: \"kubernetes.io/projected/2a94a786-51d8-4769-b320-40b215ac1147-kube-api-access-575vx\") pod \"fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.218128 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.218042    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:26.491224 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.491198    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"]
Sep 09 20:25:26.493424 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:25:26.493395    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a94a786_51d8_4769_b320_40b215ac1147.slice/crio-1854ca3293cdee3ce09d7d35d1a79101d55ec0508d33fcdb7a2d41eb2cac82ad WatchSource:0}: Error finding container 1854ca3293cdee3ce09d7d35d1a79101d55ec0508d33fcdb7a2d41eb2cac82ad: Status 404 returned error can't find the container with id 1854ca3293cdee3ce09d7d35d1a79101d55ec0508d33fcdb7a2d41eb2cac82ad
Sep 09 20:25:26.500286 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:26.496292    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:25:27.378580 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:27.378529    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" event={"ID":"2a94a786-51d8-4769-b320-40b215ac1147","Type":"ContainerStarted","Data":"1854ca3293cdee3ce09d7d35d1a79101d55ec0508d33fcdb7a2d41eb2cac82ad"}
Sep 09 20:25:29.389135 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:29.389101    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" event={"ID":"2a94a786-51d8-4769-b320-40b215ac1147","Type":"ContainerStarted","Data":"6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249"}
Sep 09 20:25:29.389525 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:29.389244    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:25:29.408431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:25:29.408380    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podStartSLOduration=2.124795648 podStartE2EDuration="4.408364182s" totalImagesPullingTime="2.283568534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:25:29.406528711 +0000 UTC m=+868.337612035" watchObservedRunningTime="2026-09-09 20:25:29.408364182 +0000 UTC m=+868.339447500"
Sep 09 20:26:00.396916 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:00.396839    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:26:01.494882 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:01.494854    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:26:01.496168 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:01.496146    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:26:01.507470 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:01.507449    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:26:01.508673 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:01.508656    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:26:02.075913 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:02.075882    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:26:10.396476 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:10.396435    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:26:10.396994 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:10.396929    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"]
Sep 09 20:26:10.406624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:10.406592    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:26:10.502928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:10.502896    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:26:20.396325 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:20.396282    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:26:25.418019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:25.417962    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main" containerID="cri-o://6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249" gracePeriod=45
Sep 09 20:26:30.396881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:30.396844    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:26:30.397362 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:30.396976    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:26:40.396387 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:40.396349    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:26:50.396901 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:50.396863    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:26:55.664566 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.664540    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:26:55.715636 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.715552    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-575vx\" (UniqueName: \"kubernetes.io/projected/2a94a786-51d8-4769-b320-40b215ac1147-kube-api-access-575vx\") pod \"2a94a786-51d8-4769-b320-40b215ac1147\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") "
Sep 09 20:26:55.715805 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.715644    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a94a786-51d8-4769-b320-40b215ac1147-tls-certs\") pod \"2a94a786-51d8-4769-b320-40b215ac1147\" (UID: \"2a94a786-51d8-4769-b320-40b215ac1147\") "
Sep 09 20:26:55.717736 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.717702    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a94a786-51d8-4769-b320-40b215ac1147-kube-api-access-575vx" (OuterVolumeSpecName: "kube-api-access-575vx") pod "2a94a786-51d8-4769-b320-40b215ac1147" (UID: "2a94a786-51d8-4769-b320-40b215ac1147"). InnerVolumeSpecName "kube-api-access-575vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:26:55.717736 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.717717    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a94a786-51d8-4769-b320-40b215ac1147-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2a94a786-51d8-4769-b320-40b215ac1147" (UID: "2a94a786-51d8-4769-b320-40b215ac1147"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:26:55.763106 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.763074    2573 generic.go:356] "Generic (PLEG): container finished" podID="2a94a786-51d8-4769-b320-40b215ac1147" containerID="6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249" exitCode=0
Sep 09 20:26:55.763280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.763120    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"
Sep 09 20:26:55.763280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.763135    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" event={"ID":"2a94a786-51d8-4769-b320-40b215ac1147","Type":"ContainerDied","Data":"6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249"}
Sep 09 20:26:55.763280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.763187    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9" event={"ID":"2a94a786-51d8-4769-b320-40b215ac1147","Type":"ContainerDied","Data":"1854ca3293cdee3ce09d7d35d1a79101d55ec0508d33fcdb7a2d41eb2cac82ad"}
Sep 09 20:26:55.763280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.763219    2573 scope.go:117] "RemoveContainer" containerID="6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249"
Sep 09 20:26:55.771881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.771859    2573 scope.go:117] "RemoveContainer" containerID="6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249"
Sep 09 20:26:55.772154 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:26:55.772137    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249\": container with ID starting with 6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249 not found: ID does not exist" containerID="6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249"
Sep 09 20:26:55.772204 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.772162    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249"} err="failed to get container status \"6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249\": rpc error: code = NotFound desc = could not find container \"6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249\": container with ID starting with 6a277d3908950426aec9c34ef73557eace526fcb9a0f11386bd0e14029301249 not found: ID does not exist"
Sep 09 20:26:55.784750 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.784729    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"]
Sep 09 20:26:55.788561 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.788540    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-76d5658f88-nf9c9"]
Sep 09 20:26:55.816529 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.816508    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a94a786-51d8-4769-b320-40b215ac1147-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:26:55.816529 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:55.816529    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-575vx\" (UniqueName: \"kubernetes.io/projected/2a94a786-51d8-4769-b320-40b215ac1147-kube-api-access-575vx\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:26:57.641686 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:26:57.641648    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a94a786-51d8-4769-b320-40b215ac1147" path="/var/lib/kubelet/pods/2a94a786-51d8-4769-b320-40b215ac1147/volumes"
Sep 09 20:27:00.498990 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:00.498941    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:01.930472 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:01.930439    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:06.825136 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:06.825096    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:08.858582 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:08.858551    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:09.019359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.019317    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"]
Sep 09 20:27:09.019719 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.019705    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main"
Sep 09 20:27:09.019779 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.019721    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main"
Sep 09 20:27:09.019814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.019790    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a94a786-51d8-4769-b320-40b215ac1147" containerName="main"
Sep 09 20:27:09.022691 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.022671    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.026468 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.026447    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Sep 09 20:27:09.026599 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.026460    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Sep 09 20:27:09.026599 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.026490    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-4pdh4\""
Sep 09 20:27:09.026599 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.026492    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Sep 09 20:27:09.026781 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.026492    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-epp-sa-dockercfg-fsj46\""
Sep 09 20:27:09.032423 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.032402    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"]
Sep 09 20:27:09.058779 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.058749    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:09.132319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.132198    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/733e15db-db57-4ad0-876e-63c760386aaf-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.132319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.132240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6mqp\" (UniqueName: \"kubernetes.io/projected/733e15db-db57-4ad0-876e-63c760386aaf-kube-api-access-n6mqp\") pod \"fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.232809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.232779    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/733e15db-db57-4ad0-876e-63c760386aaf-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.233001 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.232823    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6mqp\" (UniqueName: \"kubernetes.io/projected/733e15db-db57-4ad0-876e-63c760386aaf-kube-api-access-n6mqp\") pod \"fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.235373 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.235350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/733e15db-db57-4ad0-876e-63c760386aaf-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.241046 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.241022    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6mqp\" (UniqueName: \"kubernetes.io/projected/733e15db-db57-4ad0-876e-63c760386aaf-kube-api-access-n6mqp\") pod \"fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.336647 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.336615    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.606906 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.606883    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"]
Sep 09 20:27:09.608335 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:27:09.608308    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733e15db_db57_4ad0_876e_63c760386aaf.slice/crio-12c1c1857d0b92a64e2f6fce3ba53aca2b2d957e3c58cf2037e5933303326751 WatchSource:0}: Error finding container 12c1c1857d0b92a64e2f6fce3ba53aca2b2d957e3c58cf2037e5933303326751: Status 404 returned error can't find the container with id 12c1c1857d0b92a64e2f6fce3ba53aca2b2d957e3c58cf2037e5933303326751
Sep 09 20:27:09.827049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.827019    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" event={"ID":"733e15db-db57-4ad0-876e-63c760386aaf","Type":"ContainerStarted","Data":"2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022"}
Sep 09 20:27:09.827234 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.827066    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" event={"ID":"733e15db-db57-4ad0-876e-63c760386aaf","Type":"ContainerStarted","Data":"12c1c1857d0b92a64e2f6fce3ba53aca2b2d957e3c58cf2037e5933303326751"}
Sep 09 20:27:09.827234 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.827100    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:09.844089 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:09.844046    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podStartSLOduration=0.844025762 podStartE2EDuration="844.025762ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:27:09.84387915 +0000 UTC m=+968.774962470" watchObservedRunningTime="2026-09-09 20:27:09.844025762 +0000 UTC m=+968.775109082"
Sep 09 20:27:12.720537 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:12.720502    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:12.820476 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:12.820445    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:14.304989 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:14.304949    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:14.359116 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:14.359087    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:29.436783 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:29.436699    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:40.832564 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:40.832534    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:27:48.061779 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:48.061740    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:53.874061 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:53.874019    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:55.754987 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:55.754953    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:27:55.759756 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:55.759732    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"]
Sep 09 20:27:55.853551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:27:55.853520    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:00.833233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:00.833192    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:28:06.804756 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:06.804719    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:10.781401 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:10.781332    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main" containerID="cri-o://2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022" gracePeriod=45
Sep 09 20:28:10.832880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:10.832848    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:28:13.927691 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:13.927663    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:14.045186 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:14.045154    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:14.145979 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:14.145941    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:20.312068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:20.312031    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:20.833007 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:20.832970    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:28:20.833202 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:20.833073    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:28:26.210325 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:26.210292    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:26.313327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:26.313299    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:30.833460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:30.833422    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:28:35.341627 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:35.341593    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:35.475532 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:35.475495    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:35.589027 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:35.588993    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:35.687414 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:35.687376    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:37.469421 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:37.469392    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:37.570809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:37.570774    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:40.831973 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:28:40.831951    2573 logging.go:55] [core] [Channel #77 SubChannel #78]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.29:9003", ServerName: "10.133.0.29:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.29:9003: connect: connection refused"
Sep 09 20:28:41.021272 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.021227    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:28:41.087437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.087354    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6mqp\" (UniqueName: \"kubernetes.io/projected/733e15db-db57-4ad0-876e-63c760386aaf-kube-api-access-n6mqp\") pod \"733e15db-db57-4ad0-876e-63c760386aaf\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") "
Sep 09 20:28:41.087437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.087424    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/733e15db-db57-4ad0-876e-63c760386aaf-tls-certs\") pod \"733e15db-db57-4ad0-876e-63c760386aaf\" (UID: \"733e15db-db57-4ad0-876e-63c760386aaf\") "
Sep 09 20:28:41.089591 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.089561    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/733e15db-db57-4ad0-876e-63c760386aaf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "733e15db-db57-4ad0-876e-63c760386aaf" (UID: "733e15db-db57-4ad0-876e-63c760386aaf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:28:41.089690 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.089590    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733e15db-db57-4ad0-876e-63c760386aaf-kube-api-access-n6mqp" (OuterVolumeSpecName: "kube-api-access-n6mqp") pod "733e15db-db57-4ad0-876e-63c760386aaf" (UID: "733e15db-db57-4ad0-876e-63c760386aaf"). InnerVolumeSpecName "kube-api-access-n6mqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:28:41.187962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.187925    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6mqp\" (UniqueName: \"kubernetes.io/projected/733e15db-db57-4ad0-876e-63c760386aaf-kube-api-access-n6mqp\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:28:41.187962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.187952    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/733e15db-db57-4ad0-876e-63c760386aaf-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:28:41.231449 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.231417    2573 generic.go:356] "Generic (PLEG): container finished" podID="733e15db-db57-4ad0-876e-63c760386aaf" containerID="2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022" exitCode=0
Sep 09 20:28:41.231605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.231478    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"
Sep 09 20:28:41.231605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.231510    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" event={"ID":"733e15db-db57-4ad0-876e-63c760386aaf","Type":"ContainerDied","Data":"2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022"}
Sep 09 20:28:41.231605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.231590    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" event={"ID":"733e15db-db57-4ad0-876e-63c760386aaf","Type":"ContainerDied","Data":"12c1c1857d0b92a64e2f6fce3ba53aca2b2d957e3c58cf2037e5933303326751"}
Sep 09 20:28:41.231703 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.231615    2573 scope.go:117] "RemoveContainer" containerID="2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022"
Sep 09 20:28:41.240328 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.240312    2573 scope.go:117] "RemoveContainer" containerID="2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022"
Sep 09 20:28:41.240580 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:28:41.240563    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022\": container with ID starting with 2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022 not found: ID does not exist" containerID="2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022"
Sep 09 20:28:41.240627 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.240586    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022"} err="failed to get container status \"2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022\": rpc error: code = NotFound desc = could not find container \"2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022\": container with ID starting with 2eb91374e3f713ae0ee8650ee57e7e41f2041a72c0a5d93421aed8b69d6ed022 not found: ID does not exist"
Sep 09 20:28:41.255449 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.255426    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"]
Sep 09 20:28:41.261137 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.261116    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz"]
Sep 09 20:28:41.642437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.642407    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733e15db-db57-4ad0-876e-63c760386aaf" path="/var/lib/kubelet/pods/733e15db-db57-4ad0-876e-63c760386aaf/volumes"
Sep 09 20:28:41.831908 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:41.831864    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-88dbd48f9-k8mqz" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.29:9003\" within 1s: context deadline exceeded"
Sep 09 20:28:44.943149 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:44.943119    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:45.043117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.043067    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:45.148053 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.148026    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:45.171176 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.171145    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"]
Sep 09 20:28:45.171527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.171513    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main"
Sep 09 20:28:45.171575 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.171529    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main"
Sep 09 20:28:45.171613 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.171592    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="733e15db-db57-4ad0-876e-63c760386aaf" containerName="main"
Sep 09 20:28:45.174375 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.174357    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.176784 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.176762    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\""
Sep 09 20:28:45.176918 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.176841    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"router-gateway-1-openshift-default-dockercfg-dn6q4\""
Sep 09 20:28:45.176918 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.176854    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-gw-ca-root-cert\""
Sep 09 20:28:45.177875 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.177856    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\""
Sep 09 20:28:45.187046 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.187021    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"]
Sep 09 20:28:45.218443 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218390    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218443 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218421    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0f55f51-1407-4c47-8784-eeb55debf557-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218443 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218438    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jrt6\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-kube-api-access-5jrt6\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218455    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-credential-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218519    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-data\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218565    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0f55f51-1407-4c47-8784-eeb55debf557-istio-podinfo\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218591    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-certs\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218727 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218657    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-envoy\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.218727 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.218680    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-istio-token\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.220662 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.220643    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:45.319803 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319774    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0f55f51-1407-4c47-8784-eeb55debf557-istio-podinfo\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.319924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319815    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-certs\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.319924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319861    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-envoy\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.319924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319889    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-istio-token\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319936    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319972    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0f55f51-1407-4c47-8784-eeb55debf557-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.319998    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jrt6\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-kube-api-access-5jrt6\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-credential-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320093 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-data\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320366 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320308    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-certs\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320440 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320412    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320459    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-data\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320586 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320570    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-credential-socket\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.320742 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.320720    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0f55f51-1407-4c47-8784-eeb55debf557-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.322115 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.322092    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-envoy\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.322245 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.322227    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0f55f51-1407-4c47-8784-eeb55debf557-istio-podinfo\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.329181 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.329153    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-istio-token\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.329293 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.329278    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jrt6\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-kube-api-access-5jrt6\") pod \"router-gateway-1-openshift-default-6b46477c5b-9r4cv\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.484626 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.484548    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:45.763624 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:45.760302    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"]
Sep 09 20:28:46.255445 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:46.255408    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv" event={"ID":"a0f55f51-1407-4c47-8784-eeb55debf557","Type":"ContainerStarted","Data":"c8ba114a321a20693247a5dc4f9e357c7eb7f18007b8d586d5d87d203d0ffc01"}
Sep 09 20:28:47.456704 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:47.456660    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:48.283671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:48.283641    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:48.931706 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:48.931665    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:28:48.931961 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:48.931762    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:28:48.931961 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:48.931806    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:28:49.276083 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:49.275997    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv" event={"ID":"a0f55f51-1407-4c47-8784-eeb55debf557","Type":"ContainerStarted","Data":"7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05"}
Sep 09 20:28:49.297242 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:49.297196    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv" podStartSLOduration=1.131247045 podStartE2EDuration="4.297178381s" totalImagesPullingTime="3.165931336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:28:49.294905787 +0000 UTC m=+1068.225989103" watchObservedRunningTime="2026-09-09 20:28:49.297178381 +0000 UTC m=+1068.228261700"
Sep 09 20:28:49.485205 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:49.485172    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:49.490052 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:49.490027    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:50.280664 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:50.280634    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:50.281602 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:50.281583    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:28:51.689687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:51.689648    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:57.700500 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:57.700425    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:28:57.759839 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:28:57.759805    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:04.300387 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:04.300346    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:09.126755 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.126723    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"]
Sep 09 20:29:09.133180 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.133158    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.137037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.136799    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs\""
Sep 09 20:29:09.137037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.136812    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"default-dockercfg-zwdsn\""
Sep 09 20:29:09.138966 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.138939    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"]
Sep 09 20:29:09.176625 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.176598    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:09.223753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.223725    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.223852 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.223775    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws4sc\" (UniqueName: \"kubernetes.io/projected/de90dbcc-7041-4b14-aea7-b80f31c6530f-kube-api-access-ws4sc\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.223893 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.223842    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de90dbcc-7041-4b14-aea7-b80f31c6530f-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.223932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.223889    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.223932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.223914    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.224053 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.224032    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325245 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325219    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de90dbcc-7041-4b14-aea7-b80f31c6530f-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325411 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325273    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325411 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325384    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325530 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325482    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325530 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325509    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325634 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325531    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws4sc\" (UniqueName: \"kubernetes.io/projected/de90dbcc-7041-4b14-aea7-b80f31c6530f-kube-api-access-ws4sc\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325634 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325606    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325741 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325691    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.325854 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.325835    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.327580 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.327553    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.327845 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.327825    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de90dbcc-7041-4b14-aea7-b80f31c6530f-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.333976 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.333957    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws4sc\" (UniqueName: \"kubernetes.io/projected/de90dbcc-7041-4b14-aea7-b80f31c6530f-kube-api-access-ws4sc\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.444273 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.444188    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:09.714578 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:09.714554    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"]
Sep 09 20:29:09.716737 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:29:09.716699    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde90dbcc_7041_4b14_aea7_b80f31c6530f.slice/crio-ac5bc6965472a15f51a997e64fbb0226c747ee95ca584ba2559e6094f22a4ab2 WatchSource:0}: Error finding container ac5bc6965472a15f51a997e64fbb0226c747ee95ca584ba2559e6094f22a4ab2: Status 404 returned error can't find the container with id ac5bc6965472a15f51a997e64fbb0226c747ee95ca584ba2559e6094f22a4ab2
Sep 09 20:29:10.375096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:10.375059    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" event={"ID":"de90dbcc-7041-4b14-aea7-b80f31c6530f","Type":"ContainerStarted","Data":"ac5bc6965472a15f51a997e64fbb0226c747ee95ca584ba2559e6094f22a4ab2"}
Sep 09 20:29:11.565528 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.565435    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:11.666689 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.666597    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:11.768837 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.768795    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:11.811164 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.811134    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"]
Sep 09 20:29:11.814645 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.814622    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.817134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.817094    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"router-gateway-1-openshift-default-dockercfg-5v87g\""
Sep 09 20:29:11.817134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.817123    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Sep 09 20:29:11.817337 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.817165    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-gw-ca-root-cert\""
Sep 09 20:29:11.817337 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.817133    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Sep 09 20:29:11.827574 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.827552    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"]
Sep 09 20:29:11.891666 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.891637    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:11.948803 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.948730    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ccf596-e7f0-4299-9703-845191f7f696-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.948803 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.948789    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.948988 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.948880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-istio-token\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.948988 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.948918    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ccf596-e7f0-4299-9703-845191f7f696-istio-podinfo\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.948988 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.948944    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf4g5\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-kube-api-access-vf4g5\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.948988 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.948973    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-envoy\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.949160 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.949004    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-data\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.949160 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.949057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-certs\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.949160 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.949092    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-credential-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:11.994778 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:11.994752    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:12.050668 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050630    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-istio-token\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.050816 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050693    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ccf596-e7f0-4299-9703-845191f7f696-istio-podinfo\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.050816 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050721    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf4g5\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-kube-api-access-vf4g5\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.050816 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050755    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-envoy\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.050816 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050789    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-data\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.051050 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050818    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-certs\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.051050 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050845    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-credential-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.051050 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050894    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ccf596-e7f0-4299-9703-845191f7f696-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.051050 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.050946    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.051455 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.051427    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.052416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.052390    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-data\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.052832 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.052804    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-credential-socket\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.053114 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.053087    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-certs\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.053202 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.053109    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ccf596-e7f0-4299-9703-845191f7f696-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.060269 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.055076    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ccf596-e7f0-4299-9703-845191f7f696-istio-podinfo\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.069270 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.062463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-istio-token\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.069270 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.064469    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-envoy\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.069270 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.064761    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf4g5\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-kube-api-access-vf4g5\") pod \"router-gateway-1-openshift-default-7dcbd489bd-dd8dr\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.128724 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.128691    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:12.385003 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.384966    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" event={"ID":"de90dbcc-7041-4b14-aea7-b80f31c6530f","Type":"ContainerStarted","Data":"488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"}
Sep 09 20:29:12.407774 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.407727    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" podStartSLOduration=1.751680076 podStartE2EDuration="3.407708794s" totalImagesPullingTime="1.656028718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:29:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:29:12.406656182 +0000 UTC m=+1091.337739505" watchObservedRunningTime="2026-09-09 20:29:12.407708794 +0000 UTC m=+1091.338792115"
Sep 09 20:29:12.433010 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.432987    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"]
Sep 09 20:29:12.434493 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:29:12.434465    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ccf596_e7f0_4299_9703_845191f7f696.slice/crio-09ad7b20f8f4a439883b658240b36c582617e86b38aca219bbab7cd94c52e3ed WatchSource:0}: Error finding container 09ad7b20f8f4a439883b658240b36c582617e86b38aca219bbab7cd94c52e3ed: Status 404 returned error can't find the container with id 09ad7b20f8f4a439883b658240b36c582617e86b38aca219bbab7cd94c52e3ed
Sep 09 20:29:12.436621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.436595    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:29:12.436722 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.436650    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:29:12.436722 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.436679    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:29:12.444751 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:12.444736    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:13.391910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:13.391879    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr" event={"ID":"20ccf596-e7f0-4299-9703-845191f7f696","Type":"ContainerStarted","Data":"1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97"}
Sep 09 20:29:13.392313 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:13.391927    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr" event={"ID":"20ccf596-e7f0-4299-9703-845191f7f696","Type":"ContainerStarted","Data":"09ad7b20f8f4a439883b658240b36c582617e86b38aca219bbab7cd94c52e3ed"}
Sep 09 20:29:13.415454 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:13.415411    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr" podStartSLOduration=2.415395114 podStartE2EDuration="2.415395114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:29:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:29:13.413585169 +0000 UTC m=+1092.344668492" watchObservedRunningTime="2026-09-09 20:29:13.415395114 +0000 UTC m=+1092.346478432"
Sep 09 20:29:14.129196 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:14.129165    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:14.134506 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:14.134484    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:14.396399 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:14.396318    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:14.397203 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:14.397185    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:14.690358 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:14.690325    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:16.547088 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.546994    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"]
Sep 09 20:29:16.553202 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.553176    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"]
Sep 09 20:29:16.553674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.553411    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv" podUID="a0f55f51-1407-4c47-8784-eeb55debf557" containerName="istio-proxy" containerID="cri-o://7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05" gracePeriod=30
Sep 09 20:29:16.556359 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:16.556327    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 20:29:16.556359 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:29:16.556359 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:29:16.556359 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08" cmd=["/bin/sleep","15"]
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.556383    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" message="" err=<
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:16.556399    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" podUID="de90dbcc-7041-4b14-aea7-b80f31c6530f" containerName="main" containerID="cri-o://488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"
Sep 09 20:29:16.556559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.556439    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" podUID="de90dbcc-7041-4b14-aea7-b80f31c6530f" containerName="main" containerID="cri-o://488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08" gracePeriod=60
Sep 09 20:29:16.601370 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.601342    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:16.804823 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.804760    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:16.905572 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905533    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws4sc\" (UniqueName: \"kubernetes.io/projected/de90dbcc-7041-4b14-aea7-b80f31c6530f-kube-api-access-ws4sc\") pod \"de90dbcc-7041-4b14-aea7-b80f31c6530f\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") "
Sep 09 20:29:16.905723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905603    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-home\") pod \"de90dbcc-7041-4b14-aea7-b80f31c6530f\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") "
Sep 09 20:29:16.905723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905659    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-tmp-dir\") pod \"de90dbcc-7041-4b14-aea7-b80f31c6530f\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") "
Sep 09 20:29:16.905723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905689    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de90dbcc-7041-4b14-aea7-b80f31c6530f-tls-certs\") pod \"de90dbcc-7041-4b14-aea7-b80f31c6530f\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") "
Sep 09 20:29:16.905723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905721    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-model-cache\") pod \"de90dbcc-7041-4b14-aea7-b80f31c6530f\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") "
Sep 09 20:29:16.905972 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905754    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-dshm\") pod \"de90dbcc-7041-4b14-aea7-b80f31c6530f\" (UID: \"de90dbcc-7041-4b14-aea7-b80f31c6530f\") "
Sep 09 20:29:16.905972 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905904    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-home" (OuterVolumeSpecName: "home") pod "de90dbcc-7041-4b14-aea7-b80f31c6530f" (UID: "de90dbcc-7041-4b14-aea7-b80f31c6530f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:16.905972 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.905951    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de90dbcc-7041-4b14-aea7-b80f31c6530f" (UID: "de90dbcc-7041-4b14-aea7-b80f31c6530f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:16.906135 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.906057    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:16.906135 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.906098    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:16.906243 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.906146    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-model-cache" (OuterVolumeSpecName: "model-cache") pod "de90dbcc-7041-4b14-aea7-b80f31c6530f" (UID: "de90dbcc-7041-4b14-aea7-b80f31c6530f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:16.908148 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.908120    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-dshm" (OuterVolumeSpecName: "dshm") pod "de90dbcc-7041-4b14-aea7-b80f31c6530f" (UID: "de90dbcc-7041-4b14-aea7-b80f31c6530f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:16.908280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.908121    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de90dbcc-7041-4b14-aea7-b80f31c6530f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de90dbcc-7041-4b14-aea7-b80f31c6530f" (UID: "de90dbcc-7041-4b14-aea7-b80f31c6530f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:29:16.908280 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:16.908182    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de90dbcc-7041-4b14-aea7-b80f31c6530f-kube-api-access-ws4sc" (OuterVolumeSpecName: "kube-api-access-ws4sc") pod "de90dbcc-7041-4b14-aea7-b80f31c6530f" (UID: "de90dbcc-7041-4b14-aea7-b80f31c6530f"). InnerVolumeSpecName "kube-api-access-ws4sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:29:17.007065 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.007022    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de90dbcc-7041-4b14-aea7-b80f31c6530f-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:17.007065 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.007063    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:17.007299 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.007075    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de90dbcc-7041-4b14-aea7-b80f31c6530f-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:17.007299 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.007087    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws4sc\" (UniqueName: \"kubernetes.io/projected/de90dbcc-7041-4b14-aea7-b80f31c6530f-kube-api-access-ws4sc\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:17.411681 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.411643    2573 generic.go:356] "Generic (PLEG): container finished" podID="de90dbcc-7041-4b14-aea7-b80f31c6530f" containerID="488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08" exitCode=0
Sep 09 20:29:17.411876 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.411708    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"
Sep 09 20:29:17.411876 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.411708    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" event={"ID":"de90dbcc-7041-4b14-aea7-b80f31c6530f","Type":"ContainerDied","Data":"488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"}
Sep 09 20:29:17.411876 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.411848    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp" event={"ID":"de90dbcc-7041-4b14-aea7-b80f31c6530f","Type":"ContainerDied","Data":"ac5bc6965472a15f51a997e64fbb0226c747ee95ca584ba2559e6094f22a4ab2"}
Sep 09 20:29:17.412051 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.411879    2573 scope.go:117] "RemoveContainer" containerID="488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"
Sep 09 20:29:17.422288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.422217    2573 scope.go:117] "RemoveContainer" containerID="488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"
Sep 09 20:29:17.422928 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:17.422551    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08\": container with ID starting with 488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08 not found: ID does not exist" containerID="488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"
Sep 09 20:29:17.422928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.422590    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08"} err="failed to get container status \"488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08\": rpc error: code = NotFound desc = could not find container \"488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08\": container with ID starting with 488492e2f6bf6251daaa1859fc58c0ed8aa1e301f255e0f5b9ceea5edea50c08 not found: ID does not exist"
Sep 09 20:29:17.439479 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.439453    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"]
Sep 09 20:29:17.446238 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.446209    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-jtjzp"]
Sep 09 20:29:17.641958 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:17.641914    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de90dbcc-7041-4b14-aea7-b80f31c6530f" path="/var/lib/kubelet/pods/de90dbcc-7041-4b14-aea7-b80f31c6530f/volumes"
Sep 09 20:29:19.390301 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.390274    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:29:19.424141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.424022    2573 generic.go:356] "Generic (PLEG): container finished" podID="a0f55f51-1407-4c47-8784-eeb55debf557" containerID="7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05" exitCode=0
Sep 09 20:29:19.424141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.424073    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv" event={"ID":"a0f55f51-1407-4c47-8784-eeb55debf557","Type":"ContainerDied","Data":"7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05"}
Sep 09 20:29:19.424141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.424087    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"
Sep 09 20:29:19.424501 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.424155    2573 scope.go:117] "RemoveContainer" containerID="7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05"
Sep 09 20:29:19.424501 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.424140    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv" event={"ID":"a0f55f51-1407-4c47-8784-eeb55debf557","Type":"ContainerDied","Data":"c8ba114a321a20693247a5dc4f9e357c7eb7f18007b8d586d5d87d203d0ffc01"}
Sep 09 20:29:19.434087 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.434067    2573 scope.go:117] "RemoveContainer" containerID="7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05"
Sep 09 20:29:19.434434 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:19.434414    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05\": container with ID starting with 7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05 not found: ID does not exist" containerID="7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05"
Sep 09 20:29:19.434502 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.434444    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05"} err="failed to get container status \"7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05\": rpc error: code = NotFound desc = could not find container \"7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05\": container with ID starting with 7eadf1b99cfe55d74f339746876ab1eb1794939f21d0dad8c5163be6a7eb6d05 not found: ID does not exist"
Sep 09 20:29:19.531968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.531937    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-credential-socket\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.531968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.531976    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jrt6\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-kube-api-access-5jrt6\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.531993    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-envoy\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532038    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0f55f51-1407-4c47-8784-eeb55debf557-istiod-ca-cert\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532079    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-certs\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532148    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-socket\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532176    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-data\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532202    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0f55f51-1407-4c47-8784-eeb55debf557-istio-podinfo\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532224    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-istio-token\") pod \"a0f55f51-1407-4c47-8784-eeb55debf557\" (UID: \"a0f55f51-1407-4c47-8784-eeb55debf557\") "
Sep 09 20:29:19.532550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532295    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:19.532550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532489    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:19.532550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532526    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:19.532709 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532569    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-credential-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.532763 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532733    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:19.532824 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.532803    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0f55f51-1407-4c47-8784-eeb55debf557-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:29:19.534431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.534406    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:19.534554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.534450    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-kube-api-access-5jrt6" (OuterVolumeSpecName: "kube-api-access-5jrt6") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "kube-api-access-5jrt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:29:19.534554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.534470    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0f55f51-1407-4c47-8784-eeb55debf557-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:29:19.534554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.534511    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0f55f51-1407-4c47-8784-eeb55debf557" (UID: "a0f55f51-1407-4c47-8784-eeb55debf557"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:29:19.633651 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633617    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jrt6\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-kube-api-access-5jrt6\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633651 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633644    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-envoy\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633651 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633653    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0f55f51-1407-4c47-8784-eeb55debf557-istiod-ca-cert\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633651 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633661    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633903 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633670    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-workload-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633903 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633680    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0f55f51-1407-4c47-8784-eeb55debf557-istio-data\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633903 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633687    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0f55f51-1407-4c47-8784-eeb55debf557-istio-podinfo\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.633903 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.633698    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0f55f51-1407-4c47-8784-eeb55debf557-istio-token\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:19.742648 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.742558    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"]
Sep 09 20:29:19.747658 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:19.747630    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-6b46477c5b-9r4cv"]
Sep 09 20:29:21.643167 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:21.643133    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0f55f51-1407-4c47-8784-eeb55debf557" path="/var/lib/kubelet/pods/a0f55f51-1407-4c47-8784-eeb55debf557/volumes"
Sep 09 20:29:21.699101 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:21.699060    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:21.798771 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:21.798720    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:23.890699 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:23.890649    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:27.868833 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:27.868802    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:29.686336 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:29.686295    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:29.743703 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:29.743672    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:34.742977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.742944    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"]
Sep 09 20:29:34.743392 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.743347    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de90dbcc-7041-4b14-aea7-b80f31c6530f" containerName="main"
Sep 09 20:29:34.743392 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.743360    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="de90dbcc-7041-4b14-aea7-b80f31c6530f" containerName="main"
Sep 09 20:29:34.743392 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.743372    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0f55f51-1407-4c47-8784-eeb55debf557" containerName="istio-proxy"
Sep 09 20:29:34.743392 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.743378    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0f55f51-1407-4c47-8784-eeb55debf557" containerName="istio-proxy"
Sep 09 20:29:34.743717 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.743452    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="de90dbcc-7041-4b14-aea7-b80f31c6530f" containerName="main"
Sep 09 20:29:34.743717 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.743462    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0f55f51-1407-4c47-8784-eeb55debf557" containerName="istio-proxy"
Sep 09 20:29:34.745338 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.745320    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.748854 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.748828    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"default-dockercfg-c7vtz\""
Sep 09 20:29:34.748982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.748839    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Sep 09 20:29:34.754528 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.754506    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"]
Sep 09 20:29:34.776924 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.776896    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:34.856194 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.856170    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.856351 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.856207    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.856351 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.856317    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.856351 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.856342    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.856501 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.856408    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.856501 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.856477    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hd9f\" (UniqueName: \"kubernetes.io/projected/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-kube-api-access-2hd9f\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.957846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.957819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.957964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.957849    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.957964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.957872    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.957964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.957906    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hd9f\" (UniqueName: \"kubernetes.io/projected/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-kube-api-access-2hd9f\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.957964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.957932    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.957964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.957949    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.958284 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.958236    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.958397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.958348    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.958464 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.958439    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.960075 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.960050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.960420 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.960401    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:34.965743 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:34.965720    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hd9f\" (UniqueName: \"kubernetes.io/projected/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-kube-api-access-2hd9f\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:35.057301 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:35.057213    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:35.330976 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:35.330949    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"]
Sep 09 20:29:35.332819 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:29:35.332788    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ac76dd_81f0_4fd4_95ce_e06b3e85b80c.slice/crio-03bff82432d28e84b9d80cf090de37e88bd18baeb7b657dd4a245f981fdf27ff WatchSource:0}: Error finding container 03bff82432d28e84b9d80cf090de37e88bd18baeb7b657dd4a245f981fdf27ff: Status 404 returned error can't find the container with id 03bff82432d28e84b9d80cf090de37e88bd18baeb7b657dd4a245f981fdf27ff
Sep 09 20:29:35.499998 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:35.499963    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" event={"ID":"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c","Type":"ContainerStarted","Data":"29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"}
Sep 09 20:29:35.500176 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:35.500013    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" event={"ID":"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c","Type":"ContainerStarted","Data":"03bff82432d28e84b9d80cf090de37e88bd18baeb7b657dd4a245f981fdf27ff"}
Sep 09 20:29:35.517602 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:35.517548    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" podStartSLOduration=1.517532567 podStartE2EDuration="1.517532567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:29:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:29:35.516417091 +0000 UTC m=+1114.447500410" watchObservedRunningTime="2026-09-09 20:29:35.517532567 +0000 UTC m=+1114.448615886"
Sep 09 20:29:36.057998 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:36.057956    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:37.209458 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:37.209427    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:37.306790 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:37.306756    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:38.769948 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:38.769907    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"]
Sep 09 20:29:38.784339 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:38.784296    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:38.785801 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:38.785772    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 20:29:38.785801 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:29:38.785801 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:29:38.785801 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d" cmd=["/bin/sleep","15"]
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:38.785821    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" message="" err=<
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:38.785834    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" podUID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" containerName="main" containerID="cri-o://29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"
Sep 09 20:29:38.786026 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:38.785871    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" podUID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" containerName="main" containerID="cri-o://29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d" gracePeriod=60
Sep 09 20:29:39.079038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.079011    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:39.094431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094409    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tls-certs\") pod \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") "
Sep 09 20:29:39.094552 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094450    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-home\") pod \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") "
Sep 09 20:29:39.094552 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094483    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tmp-dir\") pod \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") "
Sep 09 20:29:39.094552 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094539    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-dshm\") pod \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") "
Sep 09 20:29:39.094658 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094572    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-model-cache\") pod \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") "
Sep 09 20:29:39.094658 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094604    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hd9f\" (UniqueName: \"kubernetes.io/projected/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-kube-api-access-2hd9f\") pod \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\" (UID: \"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c\") "
Sep 09 20:29:39.094763 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094737    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" (UID: "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:39.094810 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094772    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-home" (OuterVolumeSpecName: "home") pod "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" (UID: "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:39.094871 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094826    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-model-cache" (OuterVolumeSpecName: "model-cache") pod "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" (UID: "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:39.094922 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094892    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:39.094922 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094911    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:39.095015 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.094924    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:39.097134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.097088    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-dshm" (OuterVolumeSpecName: "dshm") pod "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" (UID: "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:39.097297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.097147    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" (UID: "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:29:39.097297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.097240    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-kube-api-access-2hd9f" (OuterVolumeSpecName: "kube-api-access-2hd9f") pod "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" (UID: "78ac76dd-81f0-4fd4-95ce-e06b3e85b80c"). InnerVolumeSpecName "kube-api-access-2hd9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:29:39.195873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.195833    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hd9f\" (UniqueName: \"kubernetes.io/projected/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-kube-api-access-2hd9f\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:39.195873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.195869    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:39.195873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.195882    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:39.519971 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.519934    2573 generic.go:356] "Generic (PLEG): container finished" podID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" containerID="29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d" exitCode=0
Sep 09 20:29:39.520165 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.519979    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"
Sep 09 20:29:39.520165 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.520020    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" event={"ID":"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c","Type":"ContainerDied","Data":"29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"}
Sep 09 20:29:39.520165 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.520076    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk" event={"ID":"78ac76dd-81f0-4fd4-95ce-e06b3e85b80c","Type":"ContainerDied","Data":"03bff82432d28e84b9d80cf090de37e88bd18baeb7b657dd4a245f981fdf27ff"}
Sep 09 20:29:39.520165 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.520098    2573 scope.go:117] "RemoveContainer" containerID="29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"
Sep 09 20:29:39.529086 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.529068    2573 scope.go:117] "RemoveContainer" containerID="29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"
Sep 09 20:29:39.529378 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:39.529358    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d\": container with ID starting with 29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d not found: ID does not exist" containerID="29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"
Sep 09 20:29:39.529433 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.529385    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d"} err="failed to get container status \"29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d\": rpc error: code = NotFound desc = could not find container \"29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d\": container with ID starting with 29c3c6a758ff9f3fcf73ffac11fffc70d0f597aaf52a4d2f2011f451bd9ec60d not found: ID does not exist"
Sep 09 20:29:39.542186 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.542157    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"]
Sep 09 20:29:39.568051 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.547613    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfxdrqk"]
Sep 09 20:29:39.642055 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:39.642020    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" path="/var/lib/kubelet/pods/78ac76dd-81f0-4fd4-95ce-e06b3e85b80c/volumes"
Sep 09 20:29:42.150912 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:42.150877    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"]
Sep 09 20:29:42.151492 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:42.151077    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr" podUID="20ccf596-e7f0-4299-9703-845191f7f696" containerName="istio-proxy" containerID="cri-o://1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97" gracePeriod=30
Sep 09 20:29:42.198772 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:42.198734    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:44.192455 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.192405    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:44.241733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241702    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-certs\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.241917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241754    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-envoy\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.241917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241781    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ccf596-e7f0-4299-9703-845191f7f696-istio-podinfo\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.241917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241811    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-credential-socket\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.241917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241847    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf4g5\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-kube-api-access-vf4g5\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.241917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241888    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-socket\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.241917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241911    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-istio-token\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.242268 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241944    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ccf596-e7f0-4299-9703-845191f7f696-istiod-ca-cert\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.242268 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241974    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-data\") pod \"20ccf596-e7f0-4299-9703-845191f7f696\" (UID: \"20ccf596-e7f0-4299-9703-845191f7f696\") "
Sep 09 20:29:44.242268 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.241976    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:44.242268 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.242205    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:44.242268 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.242212    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.242544 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.242413    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-data" (OuterVolumeSpecName: "istio-data") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:44.242636 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.242611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ccf596-e7f0-4299-9703-845191f7f696-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:29:44.242750 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.242712    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:44.244598 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.244522    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20ccf596-e7f0-4299-9703-845191f7f696-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:29:44.244910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.244877    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:29:44.244994 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.244966    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-kube-api-access-vf4g5" (OuterVolumeSpecName: "kube-api-access-vf4g5") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "kube-api-access-vf4g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:29:44.245175 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.245146    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-istio-token" (OuterVolumeSpecName: "istio-token") pod "20ccf596-e7f0-4299-9703-845191f7f696" (UID: "20ccf596-e7f0-4299-9703-845191f7f696"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:29:44.342951 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342918    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ccf596-e7f0-4299-9703-845191f7f696-istiod-ca-cert\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.342951 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342945    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-data\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.342951 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342954    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-istio-envoy\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.342951 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342961    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ccf596-e7f0-4299-9703-845191f7f696-istio-podinfo\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.343215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342970    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-credential-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.343215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342978    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf4g5\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-kube-api-access-vf4g5\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.343215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342987    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ccf596-e7f0-4299-9703-845191f7f696-workload-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.343215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.342994    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ccf596-e7f0-4299-9703-845191f7f696-istio-token\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:29:44.544012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.543922    2573 generic.go:356] "Generic (PLEG): container finished" podID="20ccf596-e7f0-4299-9703-845191f7f696" containerID="1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97" exitCode=0
Sep 09 20:29:44.544012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.543970    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"
Sep 09 20:29:44.544198 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.544012    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr" event={"ID":"20ccf596-e7f0-4299-9703-845191f7f696","Type":"ContainerDied","Data":"1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97"}
Sep 09 20:29:44.544198 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.544070    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr" event={"ID":"20ccf596-e7f0-4299-9703-845191f7f696","Type":"ContainerDied","Data":"09ad7b20f8f4a439883b658240b36c582617e86b38aca219bbab7cd94c52e3ed"}
Sep 09 20:29:44.544198 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.544093    2573 scope.go:117] "RemoveContainer" containerID="1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97"
Sep 09 20:29:44.553109 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.553089    2573 scope.go:117] "RemoveContainer" containerID="1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97"
Sep 09 20:29:44.553418 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:29:44.553399    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97\": container with ID starting with 1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97 not found: ID does not exist" containerID="1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97"
Sep 09 20:29:44.553484 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.553425    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97"} err="failed to get container status \"1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97\": rpc error: code = NotFound desc = could not find container \"1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97\": container with ID starting with 1429661c96511a6d5c93c0650b7c8eddf51cde2951bfd7d72314485c01acea97 not found: ID does not exist"
Sep 09 20:29:44.566123 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.566091    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"]
Sep 09 20:29:44.568158 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:44.568137    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-7dcbd489bd-dd8dr"]
Sep 09 20:29:45.641401 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:45.641369    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ccf596-e7f0-4299-9703-845191f7f696" path="/var/lib/kubelet/pods/20ccf596-e7f0-4299-9703-845191f7f696/volumes"
Sep 09 20:29:48.123691 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:48.123661    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:48.554970 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:48.554937    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:49.295542 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.295509    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:49.477563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.477529    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"]
Sep 09 20:29:49.477916 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.477897    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" containerName="main"
Sep 09 20:29:49.478012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.477919    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" containerName="main"
Sep 09 20:29:49.478012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.477947    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ccf596-e7f0-4299-9703-845191f7f696" containerName="istio-proxy"
Sep 09 20:29:49.478012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.477956    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ccf596-e7f0-4299-9703-845191f7f696" containerName="istio-proxy"
Sep 09 20:29:49.478164 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.478044    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ccf596-e7f0-4299-9703-845191f7f696" containerName="istio-proxy"
Sep 09 20:29:49.478164 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.478059    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ac76dd-81f0-4fd4-95ce-e06b3e85b80c" containerName="main"
Sep 09 20:29:49.480994 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.480973    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.483469 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.483444    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Sep 09 20:29:49.483574 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.483510    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Sep 09 20:29:49.484410 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.484391    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-vjpw2\""
Sep 09 20:29:49.484483 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.484425    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Sep 09 20:29:49.484483 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.484470    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-xn56p\""
Sep 09 20:29:49.491033 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.491010    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"]
Sep 09 20:29:49.546476 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.546403    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:49.589632 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.589596    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmxcs\" (UniqueName: \"kubernetes.io/projected/b99d071a-045e-47fd-93b1-aad842e72f84-kube-api-access-fmxcs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.589817 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.589705    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99d071a-045e-47fd-93b1-aad842e72f84-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.690547 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.690514    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99d071a-045e-47fd-93b1-aad842e72f84-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.690803 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.690773    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmxcs\" (UniqueName: \"kubernetes.io/projected/b99d071a-045e-47fd-93b1-aad842e72f84-kube-api-access-fmxcs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.692982 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.692961    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99d071a-045e-47fd-93b1-aad842e72f84-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.699691 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.699667    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmxcs\" (UniqueName: \"kubernetes.io/projected/b99d071a-045e-47fd-93b1-aad842e72f84-kube-api-access-fmxcs\") pod \"auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:49.791960 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:49.791935    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:50.063121 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:50.063096    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"]
Sep 09 20:29:50.065107 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:29:50.065080    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99d071a_045e_47fd_93b1_aad842e72f84.slice/crio-7eaeecdbe7b246cdc1c91d47ca230c73e5291dca18456db4aa21f32071852ea7 WatchSource:0}: Error finding container 7eaeecdbe7b246cdc1c91d47ca230c73e5291dca18456db4aa21f32071852ea7: Status 404 returned error can't find the container with id 7eaeecdbe7b246cdc1c91d47ca230c73e5291dca18456db4aa21f32071852ea7
Sep 09 20:29:50.573613 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:50.573574    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" event={"ID":"b99d071a-045e-47fd-93b1-aad842e72f84","Type":"ContainerStarted","Data":"08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b"}
Sep 09 20:29:50.573986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:50.573628    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" event={"ID":"b99d071a-045e-47fd-93b1-aad842e72f84","Type":"ContainerStarted","Data":"7eaeecdbe7b246cdc1c91d47ca230c73e5291dca18456db4aa21f32071852ea7"}
Sep 09 20:29:50.573986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:50.573655    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:29:50.590532 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:50.590478    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" podStartSLOduration=1.5904608040000001 podStartE2EDuration="1.590460804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:29:50.589605058 +0000 UTC m=+1129.520688387" watchObservedRunningTime="2026-09-09 20:29:50.590460804 +0000 UTC m=+1129.521544121"
Sep 09 20:29:54.269926 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:54.269894    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:29:58.085115 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:29:58.085066    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:30:03.342498 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:30:03.342452    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:30:03.488144 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:30:03.488109    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:30:18.338302 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:30:18.338268    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:30:21.580400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:30:21.580371    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:30:21.801437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:30:21.801403    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:30:21.902013 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:30:21.901976    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:01.523523 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:01.523334    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:31:01.527483 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:01.527455    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:31:01.537855 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:01.537836    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:31:01.540019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:01.539998    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:31:11.801758 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:11.801721    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:11.897119 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:11.897085    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:18.046134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:18.046100    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:23.835822 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:23.835790    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:23.895192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:23.895158    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:25.136589 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:25.136555    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:25.350753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:25.350722    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:37.856240 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:37.856209    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:54.862858 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:54.862782    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"]
Sep 09 20:31:54.866273 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:54.866225    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:54.922394 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:54.922367    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:59.076262 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:59.076217    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:31:59.140735 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:31:59.140700    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:01.580576 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:01.580538    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:32:09.881938 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:09.881881    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main" containerID="cri-o://08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b" gracePeriod=45
Sep 09 20:32:11.580035 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:11.579994    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:32:21.580338 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:21.580301    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:32:21.580840 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:21.580408    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:32:31.580143 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:31.580102    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:32:40.116563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.116537    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:32:40.243113 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.243040    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmxcs\" (UniqueName: \"kubernetes.io/projected/b99d071a-045e-47fd-93b1-aad842e72f84-kube-api-access-fmxcs\") pod \"b99d071a-045e-47fd-93b1-aad842e72f84\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") "
Sep 09 20:32:40.243113 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.243082    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99d071a-045e-47fd-93b1-aad842e72f84-tls-certs\") pod \"b99d071a-045e-47fd-93b1-aad842e72f84\" (UID: \"b99d071a-045e-47fd-93b1-aad842e72f84\") "
Sep 09 20:32:40.245284 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.245234    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b99d071a-045e-47fd-93b1-aad842e72f84-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b99d071a-045e-47fd-93b1-aad842e72f84" (UID: "b99d071a-045e-47fd-93b1-aad842e72f84"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:32:40.245407 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.245289    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99d071a-045e-47fd-93b1-aad842e72f84-kube-api-access-fmxcs" (OuterVolumeSpecName: "kube-api-access-fmxcs") pod "b99d071a-045e-47fd-93b1-aad842e72f84" (UID: "b99d071a-045e-47fd-93b1-aad842e72f84"). InnerVolumeSpecName "kube-api-access-fmxcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:32:40.323547 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.323518    2573 generic.go:356] "Generic (PLEG): container finished" podID="b99d071a-045e-47fd-93b1-aad842e72f84" containerID="08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b" exitCode=0
Sep 09 20:32:40.323661 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.323565    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"
Sep 09 20:32:40.323661 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.323596    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" event={"ID":"b99d071a-045e-47fd-93b1-aad842e72f84","Type":"ContainerDied","Data":"08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b"}
Sep 09 20:32:40.323661 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.323644    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv" event={"ID":"b99d071a-045e-47fd-93b1-aad842e72f84","Type":"ContainerDied","Data":"7eaeecdbe7b246cdc1c91d47ca230c73e5291dca18456db4aa21f32071852ea7"}
Sep 09 20:32:40.323821 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.323669    2573 scope.go:117] "RemoveContainer" containerID="08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b"
Sep 09 20:32:40.331597 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.331580    2573 scope.go:117] "RemoveContainer" containerID="08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b"
Sep 09 20:32:40.331865 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:32:40.331846    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b\": container with ID starting with 08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b not found: ID does not exist" containerID="08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b"
Sep 09 20:32:40.331902 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.331873    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b"} err="failed to get container status \"08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b\": rpc error: code = NotFound desc = could not find container \"08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b\": container with ID starting with 08b927b7a89ebce9e729120fac8691732a715494c53d32091a653dec30cd184b not found: ID does not exist"
Sep 09 20:32:40.343931 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.343906    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"]
Sep 09 20:32:40.344164 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.344145    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmxcs\" (UniqueName: \"kubernetes.io/projected/b99d071a-045e-47fd-93b1-aad842e72f84-kube-api-access-fmxcs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:32:40.344206 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.344167    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99d071a-045e-47fd-93b1-aad842e72f84-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:32:40.348527 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:40.348506    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-547fb7d6bb-tzxhv"]
Sep 09 20:32:41.641782 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:41.641751    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" path="/var/lib/kubelet/pods/b99d071a-045e-47fd-93b1-aad842e72f84/volumes"
Sep 09 20:32:44.955532 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:44.955496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:46.411954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:46.411916    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:49.086084 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:49.086051    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:49.186309 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:49.186277    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:51.365201 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:51.365167    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:54.999674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:54.999640    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:55.245942 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:55.245908    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:55.551158 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:55.551124    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:57.378214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:57.378178    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:32:57.478490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:32:57.478456    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:01.611958 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:01.611928    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:01.712538 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:01.712513    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:06.951858 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.951829    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"]
Sep 09 20:33:06.952211 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.952130    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main"
Sep 09 20:33:06.952211 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.952140    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main"
Sep 09 20:33:06.952211 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.952208    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b99d071a-045e-47fd-93b1-aad842e72f84" containerName="main"
Sep 09 20:33:06.956580 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.956562    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:06.958881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.958860    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\""
Sep 09 20:33:06.959827 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.959804    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Sep 09 20:33:06.959927 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.959811    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\""
Sep 09 20:33:06.959927 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.959858    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-kdkj6\""
Sep 09 20:33:06.965524 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.965506    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"]
Sep 09 20:33:06.982971 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:06.982950    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.060213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.060193    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp84h\" (UniqueName: \"kubernetes.io/projected/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-kube-api-access-jp84h\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.060329 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.060235    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.060329 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.060278    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.060329 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.060318    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.060467 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.060405    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.060467 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.060437    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.160936 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.160911    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.160936 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.160939    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161137 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.160956    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161137 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.160990    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161137 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.161014    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161137 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.161053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp84h\" (UniqueName: \"kubernetes.io/projected/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-kube-api-access-jp84h\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161402 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.161379    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161513 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.161496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.161580 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.161557    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.163332 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.163303    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.163666 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.163647    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.169486 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.169462    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp84h\" (UniqueName: \"kubernetes.io/projected/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-kube-api-access-jp84h\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.204764 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.204711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:07.267203 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.267183    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:07.535526 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:07.535491    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"]
Sep 09 20:33:07.537438 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:33:07.537412    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a9fa7a_caa4_4db9_a2cd_44b2f03eb217.slice/crio-a414d943fe523e0187c3ee9aa2eaf7974dc3dffdcc817691a59b566a53c32087 WatchSource:0}: Error finding container a414d943fe523e0187c3ee9aa2eaf7974dc3dffdcc817691a59b566a53c32087: Status 404 returned error can't find the container with id a414d943fe523e0187c3ee9aa2eaf7974dc3dffdcc817691a59b566a53c32087
Sep 09 20:33:08.450902 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:08.450868    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" event={"ID":"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217","Type":"ContainerStarted","Data":"47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"}
Sep 09 20:33:08.451302 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:08.450938    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" event={"ID":"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217","Type":"ContainerStarted","Data":"a414d943fe523e0187c3ee9aa2eaf7974dc3dffdcc817691a59b566a53c32087"}
Sep 09 20:33:08.466890 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:08.466834    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" podStartSLOduration=2.466815302 podStartE2EDuration="2.466815302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:33:08.466728838 +0000 UTC m=+1327.397812157" watchObservedRunningTime="2026-09-09 20:33:08.466815302 +0000 UTC m=+1327.397898620"
Sep 09 20:33:09.267986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:09.267955    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:16.204286 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:16.204232    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:17.268481 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:17.268448    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:17.281511 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:17.281483    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:17.500965 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:17.500934    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:32.390205 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:32.390112    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:32.804605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:32.804459    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:40.475664 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.475631    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:40.485609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.485581    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"]
Sep 09 20:33:40.495419 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:33:40.495384    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 20:33:40.495419 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:33:40.495419 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:33:40.495419 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625" cmd=["/bin/sleep","15"]
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.495436    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" message="" err=<
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:33:40.495448    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" podUID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" containerName="main" containerID="cri-o://47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"
Sep 09 20:33:40.495683 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.495482    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" podUID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" containerName="main" containerID="cri-o://47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625" gracePeriod=60
Sep 09 20:33:40.577487 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.577453    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:33:40.830927 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.830902    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:40.949076 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949037    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tls-certs\") pod \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") "
Sep 09 20:33:40.949076 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949083    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp84h\" (UniqueName: \"kubernetes.io/projected/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-kube-api-access-jp84h\") pod \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") "
Sep 09 20:33:40.949390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949137    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tmp-dir\") pod \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") "
Sep 09 20:33:40.949390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949322    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-dshm\") pod \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") "
Sep 09 20:33:40.949390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949380    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" (UID: "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:33:40.949550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949407    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-model-cache\") pod \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") "
Sep 09 20:33:40.949550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949476    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-home\") pod \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\" (UID: \"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217\") "
Sep 09 20:33:40.949736 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949688    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" (UID: "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:33:40.949808 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949743    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-home" (OuterVolumeSpecName: "home") pod "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" (UID: "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:33:40.949808 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.949760    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:33:40.951450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.951418    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" (UID: "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:33:40.951567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.951485    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-dshm" (OuterVolumeSpecName: "dshm") pod "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" (UID: "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:33:40.951567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:40.951489    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-kube-api-access-jp84h" (OuterVolumeSpecName: "kube-api-access-jp84h") pod "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" (UID: "b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217"). InnerVolumeSpecName "kube-api-access-jp84h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:33:41.051117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.051043    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:33:41.051117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.051069    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:33:41.051117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.051080    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp84h\" (UniqueName: \"kubernetes.io/projected/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-kube-api-access-jp84h\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:33:41.051117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.051090    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:33:41.051117 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.051098    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:33:41.610665 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.610631    2573 generic.go:356] "Generic (PLEG): container finished" podID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" containerID="47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625" exitCode=0
Sep 09 20:33:41.611041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.610677    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"
Sep 09 20:33:41.611041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.610714    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" event={"ID":"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217","Type":"ContainerDied","Data":"47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"}
Sep 09 20:33:41.611041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.610765    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59" event={"ID":"b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217","Type":"ContainerDied","Data":"a414d943fe523e0187c3ee9aa2eaf7974dc3dffdcc817691a59b566a53c32087"}
Sep 09 20:33:41.611041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.610789    2573 scope.go:117] "RemoveContainer" containerID="47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"
Sep 09 20:33:41.619505 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.619486    2573 scope.go:117] "RemoveContainer" containerID="47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"
Sep 09 20:33:41.619753 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:33:41.619731    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625\": container with ID starting with 47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625 not found: ID does not exist" containerID="47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"
Sep 09 20:33:41.619814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.619761    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625"} err="failed to get container status \"47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625\": rpc error: code = NotFound desc = could not find container \"47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625\": container with ID starting with 47ce444f7c7fd79dc0d5e2249e55602ae79cbf9bb0560a91b2bb81d3e89d6625 not found: ID does not exist"
Sep 09 20:33:41.630479 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.630459    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"]
Sep 09 20:33:41.641521 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:41.641495    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-668cf8f94fhjh59"]
Sep 09 20:33:43.641670 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:33:43.641641    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" path="/var/lib/kubelet/pods/b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217/volumes"
Sep 09 20:34:30.146892 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:30.146856    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:30.244915 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:30.244884    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:36.606493 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:36.606457    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:37.760340 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.760309    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"]
Sep 09 20:34:37.760685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.760640    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" containerName="main"
Sep 09 20:34:37.760685 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.760653    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" containerName="main"
Sep 09 20:34:37.760764 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.760732    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0a9fa7a-caa4-4db9-a2cd-44b2f03eb217" containerName="main"
Sep 09 20:34:37.763611 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.763593    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:37.767395 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.767376    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\""
Sep 09 20:34:37.767515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.767428    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-gqknt\""
Sep 09 20:34:37.767515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.767431    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Sep 09 20:34:37.767515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.767428    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\""
Sep 09 20:34:37.774047 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.774028    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"]
Sep 09 20:34:37.778196 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.778175    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:37.904516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.904476    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:37.904516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.904517    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:37.904718 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.904541    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:37.904718 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.904624    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-home\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:37.904718 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.904664    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rq69\" (UniqueName: \"kubernetes.io/projected/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-kube-api-access-5rq69\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:37.904718 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:37.904695    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-dshm\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.005886 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.005857    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rq69\" (UniqueName: \"kubernetes.io/projected/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-kube-api-access-5rq69\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006032 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.005896    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-dshm\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006032 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.005931    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006032 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.005961    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006032 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.005985    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006231 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.006033    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-home\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.006388    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.006406    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-home\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.006587 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.006473    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.008106 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.008085    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-dshm\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.008319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.008302    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.014123 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.014079    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rq69\" (UniqueName: \"kubernetes.io/projected/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-kube-api-access-5rq69\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.073723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.073693    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:38.095372 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.095346    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:38.342932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.342906    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"]
Sep 09 20:34:38.345031 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:34:38.344994    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c9ba0c_7499_4ae3_ad09_3b3d23e3ec48.slice/crio-ff7fc4804c84540610b3be885f8e56372f0e3e092926dd1876b9a15790bd64ec WatchSource:0}: Error finding container ff7fc4804c84540610b3be885f8e56372f0e3e092926dd1876b9a15790bd64ec: Status 404 returned error can't find the container with id ff7fc4804c84540610b3be885f8e56372f0e3e092926dd1876b9a15790bd64ec
Sep 09 20:34:38.856983 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.856940    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" event={"ID":"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48","Type":"ContainerStarted","Data":"5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"}
Sep 09 20:34:38.857410 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.857002    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" event={"ID":"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48","Type":"ContainerStarted","Data":"ff7fc4804c84540610b3be885f8e56372f0e3e092926dd1876b9a15790bd64ec"}
Sep 09 20:34:38.878601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:38.878558    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" podStartSLOduration=1.878541741 podStartE2EDuration="1.878541741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:34:38.877860363 +0000 UTC m=+1417.808943682" watchObservedRunningTime="2026-09-09 20:34:38.878541741 +0000 UTC m=+1417.809625059"
Sep 09 20:34:39.074071 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:39.074039    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:42.514943 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:42.514905    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:42.574156 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:42.574126    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:48.074485 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:48.074445    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:48.086863 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:48.086838    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:48.908039 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:48.908011    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:34:50.100535 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:50.100503    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:56.976563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:56.976462    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:34:57.072301 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:34:57.072273    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:12.082549 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.082516    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"]
Sep 09 20:35:12.087643 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.087602    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:12.091439 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:35:12.091402    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 20:35:12.091439 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:35:12.091439 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:35:12.091439 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93" cmd=["/bin/sleep","15"]
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.091454    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" message="" err=<
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:35:12.091466    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" podUID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" containerName="main" containerID="cri-o://5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"
Sep 09 20:35:12.091705 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.091497    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" podUID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" containerName="main" containerID="cri-o://5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93" gracePeriod=60
Sep 09 20:35:12.156219 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.156190    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:12.431646 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.431625    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:35:12.492545 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.492518    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tls-certs\") pod \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") "
Sep 09 20:35:12.492668 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.492550    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rq69\" (UniqueName: \"kubernetes.io/projected/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-kube-api-access-5rq69\") pod \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") "
Sep 09 20:35:12.492668 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.492573    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tmp-dir\") pod \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") "
Sep 09 20:35:12.493285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.492637    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-dshm\") pod \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") "
Sep 09 20:35:12.493285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493032    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-home\") pod \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") "
Sep 09 20:35:12.493285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493114    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-model-cache\") pod \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\" (UID: \"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48\") "
Sep 09 20:35:12.493285 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493157    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" (UID: "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:35:12.493553 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493358    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-home" (OuterVolumeSpecName: "home") pod "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" (UID: "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:35:12.493553 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493529    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" (UID: "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:35:12.495706 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493731    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:35:12.495706 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493752    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:35:12.495706 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.493773    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:35:12.500038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.500006    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-kube-api-access-5rq69" (OuterVolumeSpecName: "kube-api-access-5rq69") pod "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" (UID: "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48"). InnerVolumeSpecName "kube-api-access-5rq69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:35:12.500529 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.500391    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-dshm" (OuterVolumeSpecName: "dshm") pod "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" (UID: "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:35:12.500609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.500582    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" (UID: "b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:35:12.594835 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.594812    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:35:12.594835 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.594833    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rq69\" (UniqueName: \"kubernetes.io/projected/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-kube-api-access-5rq69\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:35:12.594958 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:12.594841    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:35:13.003917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.003884    2573 generic.go:356] "Generic (PLEG): container finished" podID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" containerID="5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93" exitCode=0
Sep 09 20:35:13.004077 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.003917    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" event={"ID":"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48","Type":"ContainerDied","Data":"5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"}
Sep 09 20:35:13.004077 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.003942    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"
Sep 09 20:35:13.004077 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.003972    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd" event={"ID":"b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48","Type":"ContainerDied","Data":"ff7fc4804c84540610b3be885f8e56372f0e3e092926dd1876b9a15790bd64ec"}
Sep 09 20:35:13.004077 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.003994    2573 scope.go:117] "RemoveContainer" containerID="5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"
Sep 09 20:35:13.013010 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.012996    2573 scope.go:117] "RemoveContainer" containerID="5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"
Sep 09 20:35:13.013280 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:35:13.013245    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93\": container with ID starting with 5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93 not found: ID does not exist" containerID="5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"
Sep 09 20:35:13.013343 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.013289    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93"} err="failed to get container status \"5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93\": rpc error: code = NotFound desc = could not find container \"5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93\": container with ID starting with 5b1f7da780685e5f60c64cc267cbf17952344bf8b583bf2c0055f61d6298dd93 not found: ID does not exist"
Sep 09 20:35:13.026244 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.026221    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"]
Sep 09 20:35:13.032702 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.032683    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-4r9pd"]
Sep 09 20:35:13.643059 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:13.643021    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" path="/var/lib/kubelet/pods/b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48/volumes"
Sep 09 20:35:41.818153 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:41.818113    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:47.089573 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:47.089541    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:47.185264 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:47.185219    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:52.296973 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.296943    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:52.491705 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.491663    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:52.681754 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.681723    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"]
Sep 09 20:35:52.682072 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.682059    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" containerName="main"
Sep 09 20:35:52.682121 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.682074    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" containerName="main"
Sep 09 20:35:52.682156 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.682129    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c9ba0c-7499-4ae3-ad09-3b3d23e3ec48" containerName="main"
Sep 09 20:35:52.685089 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.685074    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.688107 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.688079    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"kube-root-ca.crt\""
Sep 09 20:35:52.689148 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.689121    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-epp-sa-dockercfg-zj77g\""
Sep 09 20:35:52.689305 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.689158    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"openshift-service-ca.crt\""
Sep 09 20:35:52.689466 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.689453    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"default-dockercfg-tq5zs\""
Sep 09 20:35:52.689466 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.689462    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-kserve-self-signed-certs\""
Sep 09 20:35:52.699965 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.699947    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"]
Sep 09 20:35:52.703600 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.703578    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.703723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.703645    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk2fn\" (UniqueName: \"kubernetes.io/projected/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-kube-api-access-nk2fn\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.733680 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.733653    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:35:52.804497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.804476    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.804595 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.804530    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk2fn\" (UniqueName: \"kubernetes.io/projected/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-kube-api-access-nk2fn\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.806806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.806785    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.816154 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.816132    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk2fn\" (UniqueName: \"kubernetes.io/projected/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-kube-api-access-nk2fn\") pod \"auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:52.995673 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:52.995605    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:53.274015 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:53.273991    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"]
Sep 09 20:35:53.276342 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:35:53.276313    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce9ab6e_1b9b_4353_9fa2_5d314608eb63.slice/crio-b3fe691f5cfbf91e4b4cccf925ff52e0cb130a1269d9235f46787d4487162d4a WatchSource:0}: Error finding container b3fe691f5cfbf91e4b4cccf925ff52e0cb130a1269d9235f46787d4487162d4a: Status 404 returned error can't find the container with id b3fe691f5cfbf91e4b4cccf925ff52e0cb130a1269d9235f46787d4487162d4a
Sep 09 20:35:54.179579 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:54.179544    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" event={"ID":"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63","Type":"ContainerStarted","Data":"f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7"}
Sep 09 20:35:54.180002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:54.179606    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" event={"ID":"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63","Type":"ContainerStarted","Data":"b3fe691f5cfbf91e4b4cccf925ff52e0cb130a1269d9235f46787d4487162d4a"}
Sep 09 20:35:54.180002 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:54.179635    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:35:54.198969 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:54.198922    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" podStartSLOduration=2.198906409 podStartE2EDuration="2.198906409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:35:54.196482316 +0000 UTC m=+1493.127565647" watchObservedRunningTime="2026-09-09 20:35:54.198906409 +0000 UTC m=+1493.129989726"
Sep 09 20:35:54.713621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:35:54.713587    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:00.580228 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:00.580194    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:00.682490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:00.682458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:01.553114 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:01.553082    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:36:01.555858 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:01.555837    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:36:01.564542 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:01.564523    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:36:01.567429 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:01.567413    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:36:02.324390 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:02.324361    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:02.424420 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:02.424385    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:02.542314 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:02.542286    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:08.734057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:08.734022    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:14.621890 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:14.621860    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:14.721021 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:14.720989    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:15.085397 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:15.085294    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:15.137269 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:15.137230    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:19.191600 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:19.191567    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:23.205711 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:23.205678    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:23.306780 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:23.306736    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:25.184974 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:25.184903    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:36:36.593858 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:36.593828    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:56.512515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:56.512478    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:36:56.605448 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:36:56.605408    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:32.779027 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:37:32.778997    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:46.615341 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:37:46.615306    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:48.037048 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:37:48.037019    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:52.780006 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:37:52.779968    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:58.637019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:37:58.636925    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:37:58.906633 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:37:58.906519    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:01.982455 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:01.982424    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:02.252176 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:02.252090    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:11.522362 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:11.522329    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:11.571630 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:11.571596    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:18.266264 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:18.266222    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:19.606288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:19.606242    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:19.617044 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:19.617018    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"]
Sep 09 20:38:19.661703 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:19.661670    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:25.185567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:25.185531    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:38:34.636047 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:34.635988    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main" containerID="cri-o://f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7" gracePeriod=45
Sep 09 20:38:35.185481 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:35.185436    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:38:36.013896 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:36.013853    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:36.073449 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:36.073418    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:38:45.185132 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:45.185098    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:38:45.185622 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:45.185196    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:38:55.184910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:38:55.184873    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:39:04.874472 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:04.874441    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:39:04.969033 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:04.968973    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk2fn\" (UniqueName: \"kubernetes.io/projected/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-kube-api-access-nk2fn\") pod \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") "
Sep 09 20:39:04.969166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:04.969044    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-tls-certs\") pod \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\" (UID: \"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63\") "
Sep 09 20:39:04.971042 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:04.971016    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" (UID: "9ce9ab6e-1b9b-4353-9fa2-5d314608eb63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:39:04.971135 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:04.971018    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-kube-api-access-nk2fn" (OuterVolumeSpecName: "kube-api-access-nk2fn") pod "9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" (UID: "9ce9ab6e-1b9b-4353-9fa2-5d314608eb63"). InnerVolumeSpecName "kube-api-access-nk2fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:39:05.010071 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.010044    2573 generic.go:356] "Generic (PLEG): container finished" podID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerID="f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7" exitCode=0
Sep 09 20:39:05.010192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.010093    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"
Sep 09 20:39:05.010192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.010097    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" event={"ID":"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63","Type":"ContainerDied","Data":"f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7"}
Sep 09 20:39:05.010368 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.010216    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9" event={"ID":"9ce9ab6e-1b9b-4353-9fa2-5d314608eb63","Type":"ContainerDied","Data":"b3fe691f5cfbf91e4b4cccf925ff52e0cb130a1269d9235f46787d4487162d4a"}
Sep 09 20:39:05.010368 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.010244    2573 scope.go:117] "RemoveContainer" containerID="f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7"
Sep 09 20:39:05.018964 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.018950    2573 scope.go:117] "RemoveContainer" containerID="f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7"
Sep 09 20:39:05.019222 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:39:05.019204    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7\": container with ID starting with f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7 not found: ID does not exist" containerID="f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7"
Sep 09 20:39:05.019286 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.019230    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7"} err="failed to get container status \"f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7\": rpc error: code = NotFound desc = could not find container \"f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7\": container with ID starting with f405da6a6c006041d2857eef3d3c2c8eb9b68233077b25cb58c4f32f599da1e7 not found: ID does not exist"
Sep 09 20:39:05.032405 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.032387    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"]
Sep 09 20:39:05.037532 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.037512    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-596d9bb85f-fjhr9"]
Sep 09 20:39:05.070311 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.070290    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk2fn\" (UniqueName: \"kubernetes.io/projected/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-kube-api-access-nk2fn\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:39:05.070311 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.070309    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:39:05.149548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.149526    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:05.249957 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.249876    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:05.418059 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.418024    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:05.642695 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:05.642654    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" path="/var/lib/kubelet/pods/9ce9ab6e-1b9b-4353-9fa2-5d314608eb63/volumes"
Sep 09 20:39:09.623153 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:09.623113    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:09.691208 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:09.691172    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:11.789577 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.789544    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:11.953609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.953579    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"]
Sep 09 20:39:11.953919 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.953907    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main"
Sep 09 20:39:11.953960 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.953921    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main"
Sep 09 20:39:11.954000 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.953990    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce9ab6e-1b9b-4353-9fa2-5d314608eb63" containerName="main"
Sep 09 20:39:11.958358 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.958337    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:11.961838 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.961815    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"openshift-service-ca.crt\""
Sep 09 20:39:11.961838 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.961833    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"kube-root-ca.crt\""
Sep 09 20:39:11.962049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.961849    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"default-dockercfg-7fp44\""
Sep 09 20:39:11.962049 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.961880    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-canary-service-backend-7b63470c\"/\"canary-v1-kserve-self-signed-certs\""
Sep 09 20:39:11.967494 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.967472    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"]
Sep 09 20:39:11.975642 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:11.975606    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:12.027409 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.027385    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-model-cache\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.027498 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.027426    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-home\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.027539 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.027508    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-dshm\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.027574 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.027560    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70412049-c6af-4528-89e2-bf544ba2025d-tls-certs\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.027609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.027589    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-tmp-dir\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.027692 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.027615    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmsr5\" (UniqueName: \"kubernetes.io/projected/70412049-c6af-4528-89e2-bf544ba2025d-kube-api-access-hmsr5\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128423 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128399    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-home\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128456    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-dshm\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128481    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70412049-c6af-4528-89e2-bf544ba2025d-tls-certs\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-tmp-dir\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128648 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmsr5\" (UniqueName: \"kubernetes.io/projected/70412049-c6af-4528-89e2-bf544ba2025d-kube-api-access-hmsr5\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128648 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128536    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-model-cache\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128849 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128820    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-home\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.128896 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128856    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-tmp-dir\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.129005 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.128982    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-model-cache\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.130834 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.130815    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-dshm\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.131051 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.131035    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70412049-c6af-4528-89e2-bf544ba2025d-tls-certs\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.136602 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.136581    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmsr5\" (UniqueName: \"kubernetes.io/projected/70412049-c6af-4528-89e2-bf544ba2025d-kube-api-access-hmsr5\") pod \"canary-v1-kserve-65d58ccc9-n5gv6\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") " pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.269159 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.269127    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:12.542975 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.542930    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"]
Sep 09 20:39:12.545166 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:39:12.545135    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70412049_c6af_4528_89e2_bf544ba2025d.slice/crio-40dc59588db8eb316761480dec3d6617ba788f66e099434567bf837917b7d5ec WatchSource:0}: Error finding container 40dc59588db8eb316761480dec3d6617ba788f66e099434567bf837917b7d5ec: Status 404 returned error can't find the container with id 40dc59588db8eb316761480dec3d6617ba788f66e099434567bf837917b7d5ec
Sep 09 20:39:12.547008 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:12.546991    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:39:13.048271 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:13.048218    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6" event={"ID":"70412049-c6af-4528-89e2-bf544ba2025d","Type":"ContainerStarted","Data":"40dc59588db8eb316761480dec3d6617ba788f66e099434567bf837917b7d5ec"}
Sep 09 20:39:14.054384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:14.054296    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6" event={"ID":"70412049-c6af-4528-89e2-bf544ba2025d","Type":"ContainerStarted","Data":"7e7c86bc23db559eaa68856c592fbd84c772b70b7e7c8aec9a68ae61e6eaf8ba"}
Sep 09 20:39:14.071450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:14.071398    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6" podStartSLOduration=1.967754056 podStartE2EDuration="3.071381208s" totalImagesPullingTime="1.103627152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:39:14.070082286 +0000 UTC m=+1693.001165608" watchObservedRunningTime="2026-09-09 20:39:14.071381208 +0000 UTC m=+1693.002464526"
Sep 09 20:39:14.269952 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:14.269915    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:17.566531 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:17.566500    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:17.626799 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:17.626768    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:21.962496 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:21.962463    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:22.269897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:22.269817    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:22.281883 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:22.281860    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:23.108981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:23.108950    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:39:26.077417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:26.077337    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:26.175449 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:26.175414    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:32.415482 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:32.415450    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:38.087311 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:38.087244    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:38.140734 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:38.140701    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:51.521504 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.521468    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:51.710734 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.710705    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"]
Sep 09 20:39:51.714081 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.714061    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:51.717597 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.717576    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Sep 09 20:39:51.717721 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.717629    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-jqxss\""
Sep 09 20:39:51.717721 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.717576    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Sep 09 20:39:51.717721 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.717582    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-j5glf\""
Sep 09 20:39:51.717721 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.717690    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Sep 09 20:39:51.722551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.722531    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"]
Sep 09 20:39:51.733543 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.733523    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:39:51.749635 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.749613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xfdz\" (UniqueName: \"kubernetes.io/projected/471c9fde-75d1-4518-afb4-ca9e56c991b3-kube-api-access-9xfdz\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:51.749750 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.749733    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471c9fde-75d1-4518-afb4-ca9e56c991b3-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:51.850611 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.850517    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471c9fde-75d1-4518-afb4-ca9e56c991b3-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:51.850611 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.850581    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xfdz\" (UniqueName: \"kubernetes.io/projected/471c9fde-75d1-4518-afb4-ca9e56c991b3-kube-api-access-9xfdz\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:51.853043 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.853018    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471c9fde-75d1-4518-afb4-ca9e56c991b3-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:51.858391 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:51.858370    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xfdz\" (UniqueName: \"kubernetes.io/projected/471c9fde-75d1-4518-afb4-ca9e56c991b3-kube-api-access-9xfdz\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:52.024058 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:52.024028    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:52.293217 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:52.293189    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"]
Sep 09 20:39:52.295544 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:39:52.295520    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471c9fde_75d1_4518_afb4_ca9e56c991b3.slice/crio-2b8e898197af4dba493e85ba81c373d98d624f35caa69aa0a22ec35edbbdd691 WatchSource:0}: Error finding container 2b8e898197af4dba493e85ba81c373d98d624f35caa69aa0a22ec35edbbdd691: Status 404 returned error can't find the container with id 2b8e898197af4dba493e85ba81c373d98d624f35caa69aa0a22ec35edbbdd691
Sep 09 20:39:53.230671 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:53.230628    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" event={"ID":"471c9fde-75d1-4518-afb4-ca9e56c991b3","Type":"ContainerStarted","Data":"f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a"}
Sep 09 20:39:53.231061 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:53.230700    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:39:53.231061 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:53.230722    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" event={"ID":"471c9fde-75d1-4518-afb4-ca9e56c991b3","Type":"ContainerStarted","Data":"2b8e898197af4dba493e85ba81c373d98d624f35caa69aa0a22ec35edbbdd691"}
Sep 09 20:39:53.248510 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:39:53.248459    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podStartSLOduration=2.2484415110000002 podStartE2EDuration="2.248441511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:39:53.247402111 +0000 UTC m=+1732.178485429" watchObservedRunningTime="2026-09-09 20:39:53.248441511 +0000 UTC m=+1732.179524829"
Sep 09 20:40:03.868421 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:03.868390    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:24.236461 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:24.236434    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:40:31.605774 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:31.605738    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"]
Sep 09 20:40:31.619862 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:31.619831    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:31.714436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:31.714406    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:40:34.236928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:34.236888    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:40:44.236587 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:44.236551    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:40:46.626450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:46.626397    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main" containerID="cri-o://f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a" gracePeriod=45
Sep 09 20:40:54.236659 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:54.236580    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:40:54.237041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:40:54.236707    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:41:01.580055 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:01.580026    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:41:01.583673 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:01.583649    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:41:01.594933 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:01.594912    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:41:01.598063 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:01.598044    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:41:04.236274 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:04.236210    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:41:06.142364 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:06.142330    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:06.241568 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:06.241537    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:06.342782 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:06.342754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:06.460911 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:06.460840    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"]
Sep 09 20:41:06.469838 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:06.469814    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:11.230394 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:11.230361    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:14.237074 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:14.237037    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:41:16.496124 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:16.496087    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:16.875687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:16.875663    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:41:16.947834 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:16.947809    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471c9fde-75d1-4518-afb4-ca9e56c991b3-tls-certs\") pod \"471c9fde-75d1-4518-afb4-ca9e56c991b3\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") "
Sep 09 20:41:16.947973 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:16.947893    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xfdz\" (UniqueName: \"kubernetes.io/projected/471c9fde-75d1-4518-afb4-ca9e56c991b3-kube-api-access-9xfdz\") pod \"471c9fde-75d1-4518-afb4-ca9e56c991b3\" (UID: \"471c9fde-75d1-4518-afb4-ca9e56c991b3\") "
Sep 09 20:41:16.949880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:16.949851    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/471c9fde-75d1-4518-afb4-ca9e56c991b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "471c9fde-75d1-4518-afb4-ca9e56c991b3" (UID: "471c9fde-75d1-4518-afb4-ca9e56c991b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:41:16.949986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:16.949909    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/471c9fde-75d1-4518-afb4-ca9e56c991b3-kube-api-access-9xfdz" (OuterVolumeSpecName: "kube-api-access-9xfdz") pod "471c9fde-75d1-4518-afb4-ca9e56c991b3" (UID: "471c9fde-75d1-4518-afb4-ca9e56c991b3"). InnerVolumeSpecName "kube-api-access-9xfdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:41:17.049246 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.049186    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/471c9fde-75d1-4518-afb4-ca9e56c991b3-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:17.049246 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.049212    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xfdz\" (UniqueName: \"kubernetes.io/projected/471c9fde-75d1-4518-afb4-ca9e56c991b3-kube-api-access-9xfdz\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:17.602881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.602846    2573 generic.go:356] "Generic (PLEG): container finished" podID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerID="f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a" exitCode=0
Sep 09 20:41:17.603377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.602897    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"
Sep 09 20:41:17.603377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.602925    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" event={"ID":"471c9fde-75d1-4518-afb4-ca9e56c991b3","Type":"ContainerDied","Data":"f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a"}
Sep 09 20:41:17.603377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.602974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d" event={"ID":"471c9fde-75d1-4518-afb4-ca9e56c991b3","Type":"ContainerDied","Data":"2b8e898197af4dba493e85ba81c373d98d624f35caa69aa0a22ec35edbbdd691"}
Sep 09 20:41:17.603377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.602998    2573 scope.go:117] "RemoveContainer" containerID="f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a"
Sep 09 20:41:17.612008 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.611991    2573 scope.go:117] "RemoveContainer" containerID="f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a"
Sep 09 20:41:17.612231 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:41:17.612215    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a\": container with ID starting with f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a not found: ID does not exist" containerID="f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a"
Sep 09 20:41:17.612294 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.612237    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a"} err="failed to get container status \"f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a\": rpc error: code = NotFound desc = could not find container \"f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a\": container with ID starting with f1d272a3b1f10084f0d3df78793ea3c9b96e7abab174e19ca912321e4175847a not found: ID does not exist"
Sep 09 20:41:17.624698 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.624676    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"]
Sep 09 20:41:17.628895 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.628872    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-75r7z2d"]
Sep 09 20:41:17.640859 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:17.640839    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" path="/var/lib/kubelet/pods/471c9fde-75d1-4518-afb4-ca9e56c991b3/volumes"
Sep 09 20:41:21.479856 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.479811    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6" podUID="70412049-c6af-4528-89e2-bf544ba2025d" containerName="main" containerID="cri-o://7e7c86bc23db559eaa68856c592fbd84c772b70b7e7c8aec9a68ae61e6eaf8ba" gracePeriod=45
Sep 09 20:41:21.624214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.624181    2573 generic.go:356] "Generic (PLEG): container finished" podID="70412049-c6af-4528-89e2-bf544ba2025d" containerID="7e7c86bc23db559eaa68856c592fbd84c772b70b7e7c8aec9a68ae61e6eaf8ba" exitCode=0
Sep 09 20:41:21.624382 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.624273    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6" event={"ID":"70412049-c6af-4528-89e2-bf544ba2025d","Type":"ContainerDied","Data":"7e7c86bc23db559eaa68856c592fbd84c772b70b7e7c8aec9a68ae61e6eaf8ba"}
Sep 09 20:41:21.707228 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.707198    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:21.731968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.731910    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:41:21.772132 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.772102    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:21.890596 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890561    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-tmp-dir\") pod \"70412049-c6af-4528-89e2-bf544ba2025d\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") "
Sep 09 20:41:21.890596 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890599    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-model-cache\") pod \"70412049-c6af-4528-89e2-bf544ba2025d\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") "
Sep 09 20:41:21.890846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890656    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70412049-c6af-4528-89e2-bf544ba2025d-tls-certs\") pod \"70412049-c6af-4528-89e2-bf544ba2025d\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") "
Sep 09 20:41:21.890846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890693    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmsr5\" (UniqueName: \"kubernetes.io/projected/70412049-c6af-4528-89e2-bf544ba2025d-kube-api-access-hmsr5\") pod \"70412049-c6af-4528-89e2-bf544ba2025d\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") "
Sep 09 20:41:21.890846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890726    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-home\") pod \"70412049-c6af-4528-89e2-bf544ba2025d\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") "
Sep 09 20:41:21.890846 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890785    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-dshm\") pod \"70412049-c6af-4528-89e2-bf544ba2025d\" (UID: \"70412049-c6af-4528-89e2-bf544ba2025d\") "
Sep 09 20:41:21.891055 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.890914    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70412049-c6af-4528-89e2-bf544ba2025d" (UID: "70412049-c6af-4528-89e2-bf544ba2025d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:21.891157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.891048    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-model-cache" (OuterVolumeSpecName: "model-cache") pod "70412049-c6af-4528-89e2-bf544ba2025d" (UID: "70412049-c6af-4528-89e2-bf544ba2025d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:21.891157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.891064    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-home" (OuterVolumeSpecName: "home") pod "70412049-c6af-4528-89e2-bf544ba2025d" (UID: "70412049-c6af-4528-89e2-bf544ba2025d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:21.891157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.891088    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:21.893158 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.893130    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-dshm" (OuterVolumeSpecName: "dshm") pod "70412049-c6af-4528-89e2-bf544ba2025d" (UID: "70412049-c6af-4528-89e2-bf544ba2025d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:41:21.893295 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.893175    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70412049-c6af-4528-89e2-bf544ba2025d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70412049-c6af-4528-89e2-bf544ba2025d" (UID: "70412049-c6af-4528-89e2-bf544ba2025d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:41:21.893295 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.893186    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70412049-c6af-4528-89e2-bf544ba2025d-kube-api-access-hmsr5" (OuterVolumeSpecName: "kube-api-access-hmsr5") pod "70412049-c6af-4528-89e2-bf544ba2025d" (UID: "70412049-c6af-4528-89e2-bf544ba2025d"). InnerVolumeSpecName "kube-api-access-hmsr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:41:21.992030 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.991961    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:21.992030 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.991986    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:21.992030 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.991996    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70412049-c6af-4528-89e2-bf544ba2025d-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:21.992030 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.992005    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmsr5\" (UniqueName: \"kubernetes.io/projected/70412049-c6af-4528-89e2-bf544ba2025d-kube-api-access-hmsr5\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:21.992030 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:21.992014    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70412049-c6af-4528-89e2-bf544ba2025d-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:41:22.629739 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:22.629703    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6" event={"ID":"70412049-c6af-4528-89e2-bf544ba2025d","Type":"ContainerDied","Data":"40dc59588db8eb316761480dec3d6617ba788f66e099434567bf837917b7d5ec"}
Sep 09 20:41:22.629739 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:22.629735    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"
Sep 09 20:41:22.630352 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:22.629760    2573 scope.go:117] "RemoveContainer" containerID="7e7c86bc23db559eaa68856c592fbd84c772b70b7e7c8aec9a68ae61e6eaf8ba"
Sep 09 20:41:22.650980 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:22.650956    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"]
Sep 09 20:41:22.652608 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:22.652586    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-canary-service-backend-7b63470c/canary-v1-kserve-65d58ccc9-n5gv6"]
Sep 09 20:41:23.641742 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:23.641708    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70412049-c6af-4528-89e2-bf544ba2025d" path="/var/lib/kubelet/pods/70412049-c6af-4528-89e2-bf544ba2025d/volumes"
Sep 09 20:41:28.265538 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:28.265503    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:34.316384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:34.316348    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:34.381241 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:34.381213    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:36.678389 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:36.678349    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:48.780885 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:48.780852    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:50.208771 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.208733    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"]
Sep 09 20:41:50.209551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.209266    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main"
Sep 09 20:41:50.209551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.209285    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main"
Sep 09 20:41:50.209551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.209307    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70412049-c6af-4528-89e2-bf544ba2025d" containerName="main"
Sep 09 20:41:50.209551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.209315    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="70412049-c6af-4528-89e2-bf544ba2025d" containerName="main"
Sep 09 20:41:50.209551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.209398    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="471c9fde-75d1-4518-afb4-ca9e56c991b3" containerName="main"
Sep 09 20:41:50.209551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.209411    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="70412049-c6af-4528-89e2-bf544ba2025d" containerName="main"
Sep 09 20:41:50.212738 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.212714    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.216418 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.216394    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Sep 09 20:41:50.216563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.216440    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Sep 09 20:41:50.216563 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.216521    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Sep 09 20:41:50.216686 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.216562    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-spczr\""
Sep 09 20:41:50.216686 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.216670    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-dockercfg-fskb9\""
Sep 09 20:41:50.227359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.227336    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"]
Sep 09 20:41:50.274577 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.274546    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:50.321806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.321782    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.321911 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.321825    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.321911 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.321852    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-home\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.321911 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.321870    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.321911 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.321886    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvlgw\" (UniqueName: \"kubernetes.io/projected/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-kube-api-access-hvlgw\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.322056 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.321923    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422346 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422321    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422358    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-home\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422378    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422395    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvlgw\" (UniqueName: \"kubernetes.io/projected/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-kube-api-access-hvlgw\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422437    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422460    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422773 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422755    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422845 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422746    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-home\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.422845 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.422801    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.424728 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.424714    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.424869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.424854    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.435493 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.435474    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvlgw\" (UniqueName: \"kubernetes.io/projected/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-kube-api-access-hvlgw\") pod \"scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.520035 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.519962    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:50.522981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.522956    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:50.801839 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:50.801810    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"]
Sep 09 20:41:50.803736 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:41:50.803713    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d6a6b4_476a_4710_b99f_9b2ce47b1650.slice/crio-98a5fbca2b8774dbb1d91c02547bc3d0f3c1205b786f6e435b1e4f31873fd9f2 WatchSource:0}: Error finding container 98a5fbca2b8774dbb1d91c02547bc3d0f3c1205b786f6e435b1e4f31873fd9f2: Status 404 returned error can't find the container with id 98a5fbca2b8774dbb1d91c02547bc3d0f3c1205b786f6e435b1e4f31873fd9f2
Sep 09 20:41:51.756108 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:51.756075    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" event={"ID":"b9d6a6b4-476a-4710-b99f-9b2ce47b1650","Type":"ContainerStarted","Data":"98a5fbca2b8774dbb1d91c02547bc3d0f3c1205b786f6e435b1e4f31873fd9f2"}
Sep 09 20:41:52.761814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:52.761737    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" event={"ID":"b9d6a6b4-476a-4710-b99f-9b2ce47b1650","Type":"ContainerStarted","Data":"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"}
Sep 09 20:41:52.762175 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:52.761873    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:53.768284 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:53.768228    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" event={"ID":"b9d6a6b4-476a-4710-b99f-9b2ce47b1650","Type":"ContainerStarted","Data":"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"}
Sep 09 20:41:53.786103 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:53.786041    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" podStartSLOduration=2.128880126 podStartE2EDuration="3.786025037s" totalImagesPullingTime="1.657144911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:53.78544031 +0000 UTC m=+1852.716523629" watchObservedRunningTime="2026-09-09 20:41:53.786025037 +0000 UTC m=+1852.717108356"
Sep 09 20:41:54.523894 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:54.523864    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:41:54.939633 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:54.939603    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:41:55.007434 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:41:55.007386    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:00.523411 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:00.523377    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:42:00.535147 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:00.535122    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:42:00.810081 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:00.809986    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:42:01.993849 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:01.993752    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:02.094478 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:02.094443    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:02.200941 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:02.200907    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:07.074392 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:07.074349    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:11.814883 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:11.814857    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:42:14.486723 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:14.486665    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:16.463327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:16.463293    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:17.545309 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:17.545277    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:25.889794 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:25.889706    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:36.147670 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:36.147624    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:42.324312 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:42.324276    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:42.404938 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:42.404910    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:42:47.494131 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:42:47.494099    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:30.406995 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.406955    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:30.412794 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.412767    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"]
Sep 09 20:43:30.421376 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:43:30.421337    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 20:43:30.421376 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:43:30.421376 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:43:30.421376 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be" cmd=["/bin/sleep","15"]
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.421390    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" message="" err=<
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:43:30.421404    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="main" containerID="cri-o://20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"
Sep 09 20:43:30.421687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.421456    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="main" containerID="cri-o://20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be" gracePeriod=60
Sep 09 20:43:30.633012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.632981    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="llm-d-routing-sidecar" containerID="cri-o://78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de" gracePeriod=60
Sep 09 20:43:30.872343 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.872321    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:43:30.957104 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957066    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-dshm\") pod \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") "
Sep 09 20:43:30.957327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957122    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-home\") pod \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") "
Sep 09 20:43:30.957327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957140    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvlgw\" (UniqueName: \"kubernetes.io/projected/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-kube-api-access-hvlgw\") pod \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") "
Sep 09 20:43:30.957327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957174    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-model-cache\") pod \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") "
Sep 09 20:43:30.957327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957192    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tmp-dir\") pod \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") "
Sep 09 20:43:30.957327 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957225    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tls-certs\") pod \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\" (UID: \"b9d6a6b4-476a-4710-b99f-9b2ce47b1650\") "
Sep 09 20:43:30.957590 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957469    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-home" (OuterVolumeSpecName: "home") pod "b9d6a6b4-476a-4710-b99f-9b2ce47b1650" (UID: "b9d6a6b4-476a-4710-b99f-9b2ce47b1650"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:43:30.957590 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957499    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9d6a6b4-476a-4710-b99f-9b2ce47b1650" (UID: "b9d6a6b4-476a-4710-b99f-9b2ce47b1650"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:43:30.957590 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.957512    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b9d6a6b4-476a-4710-b99f-9b2ce47b1650" (UID: "b9d6a6b4-476a-4710-b99f-9b2ce47b1650"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:43:30.959486 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.959464    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-dshm" (OuterVolumeSpecName: "dshm") pod "b9d6a6b4-476a-4710-b99f-9b2ce47b1650" (UID: "b9d6a6b4-476a-4710-b99f-9b2ce47b1650"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:43:30.959572 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.959526    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-kube-api-access-hvlgw" (OuterVolumeSpecName: "kube-api-access-hvlgw") pod "b9d6a6b4-476a-4710-b99f-9b2ce47b1650" (UID: "b9d6a6b4-476a-4710-b99f-9b2ce47b1650"). InnerVolumeSpecName "kube-api-access-hvlgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:43:30.959611 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:30.959585    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9d6a6b4-476a-4710-b99f-9b2ce47b1650" (UID: "b9d6a6b4-476a-4710-b99f-9b2ce47b1650"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:43:31.057910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.057832    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:43:31.057910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.057858    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:43:31.057910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.057867    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvlgw\" (UniqueName: \"kubernetes.io/projected/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-kube-api-access-hvlgw\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:43:31.057910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.057876    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:43:31.057910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.057884    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:43:31.057910 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.057893    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d6a6b4-476a-4710-b99f-9b2ce47b1650-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:43:31.191123 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191082    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerID="20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be" exitCode=0
Sep 09 20:43:31.191123 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191112    2573 generic.go:356] "Generic (PLEG): container finished" podID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerID="78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de" exitCode=0
Sep 09 20:43:31.191366 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191143    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"
Sep 09 20:43:31.191366 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191165    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" event={"ID":"b9d6a6b4-476a-4710-b99f-9b2ce47b1650","Type":"ContainerDied","Data":"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"}
Sep 09 20:43:31.191366 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191210    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" event={"ID":"b9d6a6b4-476a-4710-b99f-9b2ce47b1650","Type":"ContainerDied","Data":"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"}
Sep 09 20:43:31.191366 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191228    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" event={"ID":"b9d6a6b4-476a-4710-b99f-9b2ce47b1650","Type":"ContainerDied","Data":"98a5fbca2b8774dbb1d91c02547bc3d0f3c1205b786f6e435b1e4f31873fd9f2"}
Sep 09 20:43:31.191366 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.191260    2573 scope.go:117] "RemoveContainer" containerID="20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"
Sep 09 20:43:31.199947 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.199927    2573 scope.go:117] "RemoveContainer" containerID="78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"
Sep 09 20:43:31.206842 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.206826    2573 scope.go:117] "RemoveContainer" containerID="20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"
Sep 09 20:43:31.207123 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:43:31.207105    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be\": container with ID starting with 20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be not found: ID does not exist" containerID="20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"
Sep 09 20:43:31.207179 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207133    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"} err="failed to get container status \"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be\": rpc error: code = NotFound desc = could not find container \"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be\": container with ID starting with 20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be not found: ID does not exist"
Sep 09 20:43:31.207179 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207149    2573 scope.go:117] "RemoveContainer" containerID="78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"
Sep 09 20:43:31.207411 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:43:31.207393    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de\": container with ID starting with 78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de not found: ID does not exist" containerID="78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"
Sep 09 20:43:31.207474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207418    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"} err="failed to get container status \"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de\": rpc error: code = NotFound desc = could not find container \"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de\": container with ID starting with 78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de not found: ID does not exist"
Sep 09 20:43:31.207474 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207432    2573 scope.go:117] "RemoveContainer" containerID="20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"
Sep 09 20:43:31.207675 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207657    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be"} err="failed to get container status \"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be\": rpc error: code = NotFound desc = could not find container \"20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be\": container with ID starting with 20465be5bcaa4b0004406c962c08599c74b4a9f1936b44e028c5c80d0726d4be not found: ID does not exist"
Sep 09 20:43:31.207714 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207677    2573 scope.go:117] "RemoveContainer" containerID="78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"
Sep 09 20:43:31.207899 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.207882    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de"} err="failed to get container status \"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de\": rpc error: code = NotFound desc = could not find container \"78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de\": container with ID starting with 78e6990799aed770598807e7581687a247df3805553cdd4b40723b9459bf38de not found: ID does not exist"
Sep 09 20:43:31.212978 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.212957    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"]
Sep 09 20:43:31.217696 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.217677    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl"]
Sep 09 20:43:31.638963 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.638929    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:31.642187 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.642163    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" path="/var/lib/kubelet/pods/b9d6a6b4-476a-4710-b99f-9b2ce47b1650/volumes"
Sep 09 20:43:31.800239 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:31.800184    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-856bdf4d56-vndzl" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8001/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 09 20:43:35.997915 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:35.997870    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:36.101063 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:36.101028    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:36.208655 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:36.208626    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:36.500781 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:36.500739    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:36.615218 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:36.615193    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:36.726515 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:36.726484    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:36.836963 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:36.836895    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:37.172923 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:37.172896    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:41.079145 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:41.079113    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:42.790193 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:42.790159    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:48.909275 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:48.909219    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:48.996773 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:48.996744    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:43:53.481518 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:43:53.481486    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:18.043379 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:18.043348    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:18.147007 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:18.146983    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:18.645145 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:18.645114    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:19.096804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:19.096727    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:20.791005 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:20.790975    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:26.020452 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:26.020415    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:26.733681 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:26.733650    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:26.850013 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:26.849978    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:32.104640 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:32.104609    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:32.192068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:32.192032    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:38.595709 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.595675    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:38.871041 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.871010    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:38.932007 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.931976    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"]
Sep 09 20:44:38.932324 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.932310    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="main"
Sep 09 20:44:38.932375 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.932325    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="main"
Sep 09 20:44:38.932375 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.932339    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="llm-d-routing-sidecar"
Sep 09 20:44:38.932375 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.932344    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="llm-d-routing-sidecar"
Sep 09 20:44:38.932470 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.932413    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="main"
Sep 09 20:44:38.932470 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.932421    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9d6a6b4-476a-4710-b99f-9b2ce47b1650" containerName="llm-d-routing-sidecar"
Sep 09 20:44:38.935316 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.935297    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:38.937711 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.937693    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Sep 09 20:44:38.937835 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.937705    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-pngj7\""
Sep 09 20:44:38.938795 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.938774    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Sep 09 20:44:38.938881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.938795    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-jw5zz\""
Sep 09 20:44:38.938881 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.938800    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Sep 09 20:44:38.943688 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.943667    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"]
Sep 09 20:44:38.978133 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.978105    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:44:38.989583 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.989560    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqcff\" (UniqueName: \"kubernetes.io/projected/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-kube-api-access-wqcff\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:38.989674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:38.989625    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:39.090691 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:39.090666    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqcff\" (UniqueName: \"kubernetes.io/projected/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-kube-api-access-wqcff\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:39.090790 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:39.090724    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:39.093134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:39.093113    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:39.100916 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:39.100897    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqcff\" (UniqueName: \"kubernetes.io/projected/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-kube-api-access-wqcff\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:39.246105 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:39.246048    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:39.515212 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:39.515108    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"]
Sep 09 20:44:39.517857 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:44:39.517834    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c1cc8a_493f_4fd5_b5c8_9bdb9ec0ab8c.slice/crio-9aa00703b030e2430e8c1c4d19e1ca2f163733ccd9ce161b075acd66dc6fa30b WatchSource:0}: Error finding container 9aa00703b030e2430e8c1c4d19e1ca2f163733ccd9ce161b075acd66dc6fa30b: Status 404 returned error can't find the container with id 9aa00703b030e2430e8c1c4d19e1ca2f163733ccd9ce161b075acd66dc6fa30b
Sep 09 20:44:40.496212 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:40.496179    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" event={"ID":"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c","Type":"ContainerStarted","Data":"61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b"}
Sep 09 20:44:40.496607 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:40.496238    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" event={"ID":"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c","Type":"ContainerStarted","Data":"9aa00703b030e2430e8c1c4d19e1ca2f163733ccd9ce161b075acd66dc6fa30b"}
Sep 09 20:44:40.512400 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:40.512356    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" podStartSLOduration=2.512341461 podStartE2EDuration="2.512341461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:44:40.511158257 +0000 UTC m=+2019.442241575" watchObservedRunningTime="2026-09-09 20:44:40.512341461 +0000 UTC m=+2019.443424778"
Sep 09 20:44:41.500948 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:41.500914    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:44:58.926481 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:44:58.926446    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:12.507371 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:12.507338    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:45:13.454981 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:13.454950    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:36.741843 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:36.741754    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:36.844469 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:36.844433    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:36.975384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:36.975350    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:41.800681 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:41.800648    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:41.903779 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:41.903736    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:43.394267 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:43.394229    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:50.617570 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:50.617526    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:45:50.751364 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:45:50.751332    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:01.609789 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:01.609756    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:46:01.619403 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:01.619376    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:46:01.628316 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:01.628292    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:46:01.631588 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:01.631569    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:46:08.127300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:08.127236    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:19.624768 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:19.624731    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:26.132497 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:26.132455    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:26.230994 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:26.230956    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.694462 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:45.694431    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:45.729154 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:45.729124    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"]
Sep 09 20:46:45.893395 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:45.893360    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:46:52.507529 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:52.507493    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:46:58.704809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:46:58.704727    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:00.747644 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:00.747579    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main" containerID="cri-o://61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b" gracePeriod=45
Sep 09 20:47:02.507468 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:02.507432    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:47:12.507178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:12.507141    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:47:12.507599 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:12.507242    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:47:14.981506 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:14.981471    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:15.081588 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:15.081555    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:15.184491 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:15.184461    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:15.279661 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:15.279581    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:20.062293 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:20.062233    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:21.294722 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:21.294691    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:22.507730 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:22.507680    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:47:30.396359 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:30.396302    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:30.995069 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:30.995045    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:47:31.095873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.095805    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-tls-certs\") pod \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") "
Sep 09 20:47:31.095873 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.095868    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqcff\" (UniqueName: \"kubernetes.io/projected/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-kube-api-access-wqcff\") pod \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\" (UID: \"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c\") "
Sep 09 20:47:31.097965 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.097933    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-kube-api-access-wqcff" (OuterVolumeSpecName: "kube-api-access-wqcff") pod "94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" (UID: "94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c"). InnerVolumeSpecName "kube-api-access-wqcff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:47:31.098066 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.097983    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" (UID: "94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:47:31.197221 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.197194    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqcff\" (UniqueName: \"kubernetes.io/projected/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-kube-api-access-wqcff\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.197356 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.197218    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:47:31.251137 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.251112    2573 generic.go:356] "Generic (PLEG): container finished" podID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerID="61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b" exitCode=0
Sep 09 20:47:31.251303 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.251169    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"
Sep 09 20:47:31.251303 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.251193    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" event={"ID":"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c","Type":"ContainerDied","Data":"61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b"}
Sep 09 20:47:31.251431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.251319    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf" event={"ID":"94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c","Type":"ContainerDied","Data":"9aa00703b030e2430e8c1c4d19e1ca2f163733ccd9ce161b075acd66dc6fa30b"}
Sep 09 20:47:31.251431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.251357    2573 scope.go:117] "RemoveContainer" containerID="61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b"
Sep 09 20:47:31.259991 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.259975    2573 scope.go:117] "RemoveContainer" containerID="61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b"
Sep 09 20:47:31.260279 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:47:31.260235    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b\": container with ID starting with 61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b not found: ID does not exist" containerID="61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b"
Sep 09 20:47:31.260376 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.260287    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b"} err="failed to get container status \"61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b\": rpc error: code = NotFound desc = could not find container \"61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b\": container with ID starting with 61c268505ec49a1a1ca5a4af32c038e0fd6b0e7a6abca1df46c39cce5328f48b not found: ID does not exist"
Sep 09 20:47:31.274704 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.274677    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"]
Sep 09 20:47:31.279143 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.279118    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-58494b692q2pf"]
Sep 09 20:47:31.642715 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:31.642684    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" path="/var/lib/kubelet/pods/94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c/volumes"
Sep 09 20:47:34.126084 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:34.126034    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:34.216498 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:34.216460    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:36.543790 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:36.543759    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:41.028479 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:41.028445    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:42.515300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:42.515261    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:42.618493 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:42.618454    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:46.743019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:46.742981    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:46.842906 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:46.842871    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:56.652001 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.651967    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:56.868905 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.868876    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"]
Sep 09 20:47:56.869219 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.869205    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main"
Sep 09 20:47:56.869277 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.869221    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main"
Sep 09 20:47:56.869319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.869300    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="94c1cc8a-493f-4fd5-b5c8-9bdb9ec0ab8c" containerName="main"
Sep 09 20:47:56.872304 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.872284    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:56.874812 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.874784    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-nfjpv\""
Sep 09 20:47:56.874954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.874837    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-h65sj\""
Sep 09 20:47:56.874954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.874835    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Sep 09 20:47:56.874954 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.874924    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Sep 09 20:47:56.875176 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.875054    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Sep 09 20:47:56.881181 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.881161    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"]
Sep 09 20:47:56.892904 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.892880    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:56.975110 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:56.975035    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:47:57.005729 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.005706    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.005832 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.005736    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzbsq\" (UniqueName: \"kubernetes.io/projected/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-kube-api-access-nzbsq\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.106520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.106492    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.106635 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.106533    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzbsq\" (UniqueName: \"kubernetes.io/projected/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-kube-api-access-nzbsq\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.109038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.109017    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.115897 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.115878    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzbsq\" (UniqueName: \"kubernetes.io/projected/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-kube-api-access-nzbsq\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.183455 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.183429    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:57.455244 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:57.455222    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"]
Sep 09 20:47:58.376824 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:58.376775    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" event={"ID":"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd","Type":"ContainerStarted","Data":"6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738"}
Sep 09 20:47:58.377212 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:58.376844    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" event={"ID":"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd","Type":"ContainerStarted","Data":"f7951cd7bcc537a7e0a0aab8edaae478d1b937b9ad4d3489c03653324a63e724"}
Sep 09 20:47:58.377212 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:58.376971    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:47:58.393963 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:47:58.393918    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podStartSLOduration=2.393905574 podStartE2EDuration="2.393905574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:58.392491639 +0000 UTC m=+2217.323574957" watchObservedRunningTime="2026-09-09 20:47:58.393905574 +0000 UTC m=+2217.324988891"
Sep 09 20:48:04.810742 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:04.810711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:10.853971 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:10.853941    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:10.954436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:10.954406    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:11.054368 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:11.054343    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:14.160475 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:14.160444    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:14.954476 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:14.954444    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:15.909106 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:15.909072    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:23.116698 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:23.116665    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:29.383490 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:29.383416    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:48:31.457804 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:31.457774    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:51.380463 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:51.380427    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:53.618511 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:53.618481    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:59.546520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:59.546487    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:48:59.612496 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:48:59.612457    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:00.887595 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:00.887559    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:29.090906 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:29.090871    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:29.091419 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:29.090968    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"]
Sep 09 20:49:29.147575 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:29.147540    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:29.383300 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:29.383236    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:49:39.383379 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:39.383337    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:49:44.110733 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:44.110686    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main" containerID="cri-o://6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738" gracePeriod=45
Sep 09 20:49:49.383227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:49.383184    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:49:49.383655 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:49.383330    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:49:59.382773 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:49:59.382735    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:50:08.609675 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:08.609633    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:08.712020 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:08.711983    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:08.794745 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:08.794710    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:09.383024 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:09.382986    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:50:09.605293 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:09.605228    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:09.954517 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:09.954480    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:13.682654 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:13.682623    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:14.351604 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.351579    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:50:14.440534 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.440464    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzbsq\" (UniqueName: \"kubernetes.io/projected/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-kube-api-access-nzbsq\") pod \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") "
Sep 09 20:50:14.440654 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.440548    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-tls-certs\") pod \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\" (UID: \"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd\") "
Sep 09 20:50:14.442580 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.442557    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-kube-api-access-nzbsq" (OuterVolumeSpecName: "kube-api-access-nzbsq") pod "8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" (UID: "8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd"). InnerVolumeSpecName "kube-api-access-nzbsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:50:14.442681 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.442606    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" (UID: "8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:50:14.541577 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.541549    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:50:14.541577 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.541573    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzbsq\" (UniqueName: \"kubernetes.io/projected/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd-kube-api-access-nzbsq\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:50:14.988085 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.988049    2573 generic.go:356] "Generic (PLEG): container finished" podID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerID="6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738" exitCode=0
Sep 09 20:50:14.988503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.988097    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"
Sep 09 20:50:14.988503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.988132    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" event={"ID":"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd","Type":"ContainerDied","Data":"6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738"}
Sep 09 20:50:14.988503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.988185    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22" event={"ID":"8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd","Type":"ContainerDied","Data":"f7951cd7bcc537a7e0a0aab8edaae478d1b937b9ad4d3489c03653324a63e724"}
Sep 09 20:50:14.988503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.988207    2573 scope.go:117] "RemoveContainer" containerID="6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738"
Sep 09 20:50:14.996902 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.996885    2573 scope.go:117] "RemoveContainer" containerID="6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738"
Sep 09 20:50:14.997154 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:50:14.997137    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738\": container with ID starting with 6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738 not found: ID does not exist" containerID="6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738"
Sep 09 20:50:14.997202 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:14.997162    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738"} err="failed to get container status \"6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738\": rpc error: code = NotFound desc = could not find container \"6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738\": container with ID starting with 6125d70a304c059105b7ed143c1dc24aa8bd602f5ffaf5508736a959fc4c7738 not found: ID does not exist"
Sep 09 20:50:15.009748 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:15.009726    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"]
Sep 09 20:50:15.013801 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:15.013780    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5785ffn22"]
Sep 09 20:50:15.642443 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:15.642412    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" path="/var/lib/kubelet/pods/8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd/volumes"
Sep 09 20:50:15.722921 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:15.722891    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:15.850369 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:15.850339    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:21.272724 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:21.272691    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:22.428953 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:22.428918    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:28.266986 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:28.266946    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:28.367509 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:28.367472    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:41.400235 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:41.400203    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:50.817499 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:50.817461    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:56.677809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:56.677775    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:56.741413 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:56.741381    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:58.075092 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.075061    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:58.301384 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.301350    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"]
Sep 09 20:50:58.301673 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.301662    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main"
Sep 09 20:50:58.301720 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.301674    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main"
Sep 09 20:50:58.301755 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.301730    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fe6f5f4-5cf7-4ca3-9c03-e857c265f0fd" containerName="main"
Sep 09 20:50:58.304767 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.304745    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.307150 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.307130    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Sep 09 20:50:58.308325 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.308300    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-k776l\""
Sep 09 20:50:58.308451 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.308323    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-epp-sa-dockercfg-8lfd5\""
Sep 09 20:50:58.308451 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.308310    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Sep 09 20:50:58.308451 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.308303    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Sep 09 20:50:58.314238 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.314215    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"]
Sep 09 20:50:58.391876 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.391851    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p2v7\" (UniqueName: \"kubernetes.io/projected/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-kube-api-access-8p2v7\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.391995 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.391883    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.492247 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.492218    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p2v7\" (UniqueName: \"kubernetes.io/projected/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-kube-api-access-8p2v7\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.492353 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.492275    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.494691 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.494669    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.500718 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.500695    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p2v7\" (UniqueName: \"kubernetes.io/projected/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-kube-api-access-8p2v7\") pod \"tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.503067 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.503046    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:58.597460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.597437    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:58.616350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.616327    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:58.886425 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:58.886401    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"]
Sep 09 20:50:58.889100 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:50:58.889075    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e32f775_2d54_4ac4_81b2_e1a8dd3493a8.slice/crio-6f020cc1edc7bff55f6ee9e55646c95a828ab70bd579b2cc808ad846493fbe73 WatchSource:0}: Error finding container 6f020cc1edc7bff55f6ee9e55646c95a828ab70bd579b2cc808ad846493fbe73: Status 404 returned error can't find the container with id 6f020cc1edc7bff55f6ee9e55646c95a828ab70bd579b2cc808ad846493fbe73
Sep 09 20:50:59.178152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:59.178117    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" event={"ID":"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8","Type":"ContainerStarted","Data":"0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee"}
Sep 09 20:50:59.178565 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:59.178166    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" event={"ID":"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8","Type":"ContainerStarted","Data":"6f020cc1edc7bff55f6ee9e55646c95a828ab70bd579b2cc808ad846493fbe73"}
Sep 09 20:50:59.178565 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:59.178216    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:50:59.200265 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:50:59.200170    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" podStartSLOduration=1.200156547 podStartE2EDuration="1.200156547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:59.198524643 +0000 UTC m=+2398.129607966" watchObservedRunningTime="2026-09-09 20:50:59.200156547 +0000 UTC m=+2398.131239866"
Sep 09 20:51:01.645166 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:01.645136    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:51:01.648799 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:01.648770    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:51:01.660186 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:01.660162    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:51:01.663433 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:01.663404    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:51:02.873530 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:02.873497    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:13.806869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:13.806833    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:30.184701 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:30.184620    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:51:34.674123 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:34.674091    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:53.758076 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:53.758034    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:51:53.781843 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:53.781813    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"]
Sep 09 20:51:53.829287 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:51:53.829236    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:00.184572 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:00.184534    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:52:08.802233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:08.802171    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main" containerID="cri-o://0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee" gracePeriod=45
Sep 09 20:52:10.184528 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:10.184491    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:52:19.800228 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:19.800192    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:19.903750 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:19.903719    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:20.001719 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:20.001688    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:20.185057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:20.185021    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:52:20.185244 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:20.185129    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:52:20.374003 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:20.373971    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:21.164177 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:21.164143    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:24.880582 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:24.880545    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:30.184493 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:30.184453    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 20:52:38.208990 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:38.208954    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:39.045970 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.045946    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:52:39.075732 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.075649    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-tls-certs\") pod \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") "
Sep 09 20:52:39.075732 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.075687    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p2v7\" (UniqueName: \"kubernetes.io/projected/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-kube-api-access-8p2v7\") pod \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\" (UID: \"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8\") "
Sep 09 20:52:39.077916 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.077878    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-kube-api-access-8p2v7" (OuterVolumeSpecName: "kube-api-access-8p2v7") pod "0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" (UID: "0e32f775-2d54-4ac4-81b2-e1a8dd3493a8"). InnerVolumeSpecName "kube-api-access-8p2v7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:52:39.077916 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.077903    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" (UID: "0e32f775-2d54-4ac4-81b2-e1a8dd3493a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:52:39.177194 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.177164    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:52:39.177194 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.177190    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p2v7\" (UniqueName: \"kubernetes.io/projected/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8-kube-api-access-8p2v7\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:52:39.613294 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.613232    2573 generic.go:356] "Generic (PLEG): container finished" podID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerID="0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee" exitCode=0
Sep 09 20:52:39.613294 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.613296    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"
Sep 09 20:52:39.613806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.613320    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" event={"ID":"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8","Type":"ContainerDied","Data":"0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee"}
Sep 09 20:52:39.613806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.613394    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx" event={"ID":"0e32f775-2d54-4ac4-81b2-e1a8dd3493a8","Type":"ContainerDied","Data":"6f020cc1edc7bff55f6ee9e55646c95a828ab70bd579b2cc808ad846493fbe73"}
Sep 09 20:52:39.613806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.613415    2573 scope.go:117] "RemoveContainer" containerID="0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee"
Sep 09 20:52:39.622157 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.622142    2573 scope.go:117] "RemoveContainer" containerID="0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee"
Sep 09 20:52:39.622438 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:52:39.622421    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee\": container with ID starting with 0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee not found: ID does not exist" containerID="0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee"
Sep 09 20:52:39.622507 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.622449    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee"} err="failed to get container status \"0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee\": rpc error: code = NotFound desc = could not find container \"0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee\": container with ID starting with 0a10698c31e5cf2049db129a6e0dea2cfad69288efdb74f9531ede70fdea79ee not found: ID does not exist"
Sep 09 20:52:39.642192 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.642068    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"]
Sep 09 20:52:39.643867 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:39.643847    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-8cd7dnhbcx"]
Sep 09 20:52:41.642183 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:41.642148    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" path="/var/lib/kubelet/pods/0e32f775-2d54-4ac4-81b2-e1a8dd3493a8/volumes"
Sep 09 20:52:42.241431 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:42.241398    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:42.339955 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:42.339925    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:43.038132 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:43.038092    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:43.139484 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:43.139455    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:50.303809 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:50.303763    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:56.288609 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:56.288528    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:56.391061 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:56.391025    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:59.649709 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.649680    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"]
Sep 09 20:52:59.650072 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.649989    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main"
Sep 09 20:52:59.650072 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.650000    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main"
Sep 09 20:52:59.650072 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.650071    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e32f775-2d54-4ac4-81b2-e1a8dd3493a8" containerName="main"
Sep 09 20:52:59.654548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.654520    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.656937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.656917    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"kube-root-ca.crt\""
Sep 09 20:52:59.657882 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.657860    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"openshift-service-ca.crt\""
Sep 09 20:52:59.657989 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.657891    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"default-dockercfg-gf9tw\""
Sep 09 20:52:59.658056 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.657989    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-delete-at-nonzero-weight-6b4017d2\"/\"delw-v2-kserve-self-signed-certs\""
Sep 09 20:52:59.666582 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.666561    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"]
Sep 09 20:52:59.690018 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.689979    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:59.743213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.743174    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-home\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.743213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.743205    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-model-cache\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.743484 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.743224    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l4k9\" (UniqueName: \"kubernetes.io/projected/8fccd617-d724-4aca-b479-ef78c7565bfc-kube-api-access-5l4k9\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.743484 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.743308    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-tmp-dir\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.743484 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.743369    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fccd617-d724-4aca-b479-ef78c7565bfc-tls-certs\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.743484 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.743411    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-dshm\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.844687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.844648    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-home\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.844687 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.844690    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-model-cache\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.844893 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.844707    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l4k9\" (UniqueName: \"kubernetes.io/projected/8fccd617-d724-4aca-b479-ef78c7565bfc-kube-api-access-5l4k9\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.844893 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.844740    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-tmp-dir\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.844893 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.844784    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fccd617-d724-4aca-b479-ef78c7565bfc-tls-certs\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.844893 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.844836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-dshm\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.845083 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.845055    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-home\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.845136 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.845096    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-model-cache\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.845198 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.845180    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-tmp-dir\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.846859 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.846829    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-dshm\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.847083 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.847067    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fccd617-d724-4aca-b479-ef78c7565bfc-tls-certs\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.859605 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.859581    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l4k9\" (UniqueName: \"kubernetes.io/projected/8fccd617-d724-4aca-b479-ef78c7565bfc-kube-api-access-5l4k9\") pod \"delw-v2-kserve-5b84cc7ddf-5jwz4\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") " pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:52:59.964439 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:52:59.964355    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:00.235229 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:00.235203    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"]
Sep 09 20:53:00.237701 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:53:00.237672    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fccd617_d724_4aca_b479_ef78c7565bfc.slice/crio-9552c4a5a8659b08c8e1cd96cc8c70f166c5f77ca6dde4935fa3370413226fe7 WatchSource:0}: Error finding container 9552c4a5a8659b08c8e1cd96cc8c70f166c5f77ca6dde4935fa3370413226fe7: Status 404 returned error can't find the container with id 9552c4a5a8659b08c8e1cd96cc8c70f166c5f77ca6dde4935fa3370413226fe7
Sep 09 20:53:00.704768 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:00.704728    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" event={"ID":"8fccd617-d724-4aca-b479-ef78c7565bfc","Type":"ContainerStarted","Data":"a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c"}
Sep 09 20:53:00.705287 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:00.704800    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" event={"ID":"8fccd617-d724-4aca-b479-ef78c7565bfc","Type":"ContainerStarted","Data":"9552c4a5a8659b08c8e1cd96cc8c70f166c5f77ca6dde4935fa3370413226fe7"}
Sep 09 20:53:00.722171 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:00.722124    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" podStartSLOduration=1.7221089040000002 podStartE2EDuration="1.722108904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:00.720650813 +0000 UTC m=+2519.651734130" watchObservedRunningTime="2026-09-09 20:53:00.722108904 +0000 UTC m=+2519.653192221"
Sep 09 20:53:00.965405 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:00.965317    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:02.610186 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:02.610154    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:06.413550 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:06.413498    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:06.501746 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:06.501712    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.727436 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:08.727396    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.418872 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:09.418833    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.516885 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:09.516854    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.965085 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:09.965060    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:09.977064 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:09.977041    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:10.760091 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:10.760065    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:28.495178 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:28.495144    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.735761 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:39.735717    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.834995 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:39.834966    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:41.605681 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:41.605652    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"]
Sep 09 20:53:41.629782 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:41.629753    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:56.624375 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.624317    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" podUID="8fccd617-d724-4aca-b479-ef78c7565bfc" containerName="main" containerID="cri-o://a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c" gracePeriod=45
Sep 09 20:53:56.750141 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.750108    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" podUID="8fccd617-d724-4aca-b479-ef78c7565bfc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/health\": dial tcp 10.133.0.44:8000: connect: connection refused"
Sep 09 20:53:56.864878 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.864857    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:56.946908 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.946835    2573 generic.go:356] "Generic (PLEG): container finished" podID="8fccd617-d724-4aca-b479-ef78c7565bfc" containerID="a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c" exitCode=0
Sep 09 20:53:56.946908 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.946875    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"
Sep 09 20:53:56.947068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.946919    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" event={"ID":"8fccd617-d724-4aca-b479-ef78c7565bfc","Type":"ContainerDied","Data":"a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c"}
Sep 09 20:53:56.947068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.946974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4" event={"ID":"8fccd617-d724-4aca-b479-ef78c7565bfc","Type":"ContainerDied","Data":"9552c4a5a8659b08c8e1cd96cc8c70f166c5f77ca6dde4935fa3370413226fe7"}
Sep 09 20:53:56.947068 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.947000    2573 scope.go:117] "RemoveContainer" containerID="a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c"
Sep 09 20:53:56.954736 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.954720    2573 scope.go:117] "RemoveContainer" containerID="a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c"
Sep 09 20:53:56.955004 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:53:56.954984    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c\": container with ID starting with a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c not found: ID does not exist" containerID="a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c"
Sep 09 20:53:56.955103 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.955011    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c"} err="failed to get container status \"a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c\": rpc error: code = NotFound desc = could not find container \"a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c\": container with ID starting with a25394e9d7ad5f93c345a3327f4db94de35eaf8d44eb6e355b8c533449bbce0c not found: ID does not exist"
Sep 09 20:53:56.992006 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.991985    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l4k9\" (UniqueName: \"kubernetes.io/projected/8fccd617-d724-4aca-b479-ef78c7565bfc-kube-api-access-5l4k9\") pod \"8fccd617-d724-4aca-b479-ef78c7565bfc\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") "
Sep 09 20:53:56.992096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992011    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-tmp-dir\") pod \"8fccd617-d724-4aca-b479-ef78c7565bfc\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") "
Sep 09 20:53:56.992096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992028    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-dshm\") pod \"8fccd617-d724-4aca-b479-ef78c7565bfc\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") "
Sep 09 20:53:56.992096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992065    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-model-cache\") pod \"8fccd617-d724-4aca-b479-ef78c7565bfc\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") "
Sep 09 20:53:56.992281 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992105    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fccd617-d724-4aca-b479-ef78c7565bfc-tls-certs\") pod \"8fccd617-d724-4aca-b479-ef78c7565bfc\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") "
Sep 09 20:53:56.992281 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992134    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-home\") pod \"8fccd617-d724-4aca-b479-ef78c7565bfc\" (UID: \"8fccd617-d724-4aca-b479-ef78c7565bfc\") "
Sep 09 20:53:56.992387 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992304    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8fccd617-d724-4aca-b479-ef78c7565bfc" (UID: "8fccd617-d724-4aca-b479-ef78c7565bfc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:56.992387 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992370    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-model-cache" (OuterVolumeSpecName: "model-cache") pod "8fccd617-d724-4aca-b479-ef78c7565bfc" (UID: "8fccd617-d724-4aca-b479-ef78c7565bfc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:56.992489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992395    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.992549 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.992503    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-home" (OuterVolumeSpecName: "home") pod "8fccd617-d724-4aca-b479-ef78c7565bfc" (UID: "8fccd617-d724-4aca-b479-ef78c7565bfc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:56.994136 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.994113    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-dshm" (OuterVolumeSpecName: "dshm") pod "8fccd617-d724-4aca-b479-ef78c7565bfc" (UID: "8fccd617-d724-4aca-b479-ef78c7565bfc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:56.994214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.994154    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fccd617-d724-4aca-b479-ef78c7565bfc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8fccd617-d724-4aca-b479-ef78c7565bfc" (UID: "8fccd617-d724-4aca-b479-ef78c7565bfc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:53:56.994214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:56.994184    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fccd617-d724-4aca-b479-ef78c7565bfc-kube-api-access-5l4k9" (OuterVolumeSpecName: "kube-api-access-5l4k9") pod "8fccd617-d724-4aca-b479-ef78c7565bfc" (UID: "8fccd617-d724-4aca-b479-ef78c7565bfc"). InnerVolumeSpecName "kube-api-access-5l4k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:53:57.093131 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.093111    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:53:57.093131 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.093131    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fccd617-d724-4aca-b479-ef78c7565bfc-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:53:57.093283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.093140    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:53:57.093283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.093148    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l4k9\" (UniqueName: \"kubernetes.io/projected/8fccd617-d724-4aca-b479-ef78c7565bfc-kube-api-access-5l4k9\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:53:57.093283 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.093156    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fccd617-d724-4aca-b479-ef78c7565bfc-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:53:57.269949 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.269924    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"]
Sep 09 20:53:57.274193 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.274163    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-delete-at-nonzero-weight-6b4017d2/delw-v2-kserve-5b84cc7ddf-5jwz4"]
Sep 09 20:53:57.492672 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.492633    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:57.641998 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:57.641969    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fccd617-d724-4aca-b479-ef78c7565bfc" path="/var/lib/kubelet/pods/8fccd617-d724-4aca-b479-ef78c7565bfc/volumes"
Sep 09 20:53:59.079641 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:53:59.079611    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:00.350040 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.350010    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"]
Sep 09 20:54:00.350513 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.350346    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fccd617-d724-4aca-b479-ef78c7565bfc" containerName="main"
Sep 09 20:54:00.350513 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.350356    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fccd617-d724-4aca-b479-ef78c7565bfc" containerName="main"
Sep 09 20:54:00.350513 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.350422    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fccd617-d724-4aca-b479-ef78c7565bfc" containerName="main"
Sep 09 20:54:00.354758 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.354738    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.358505 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.358483    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"kube-root-ca.crt\""
Sep 09 20:54:00.358621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.358483    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"openshift-service-ca.crt\""
Sep 09 20:54:00.358621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.358555    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"roll-v1-kserve-self-signed-certs\""
Sep 09 20:54:00.358621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.358605    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rollback-b8fdb441\"/\"default-dockercfg-7s76d\""
Sep 09 20:54:00.364621 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.364601    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"]
Sep 09 20:54:00.376637 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.376616    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:00.420705 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.420682    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-dshm\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.420837 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.420716    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-home\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.420837 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.420736    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf5jb\" (UniqueName: \"kubernetes.io/projected/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-kube-api-access-qf5jb\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.420837 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.420753    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tls-certs\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.420977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.420867    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tmp-dir\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.420977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.420915    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-model-cache\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522020 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.521987    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-dshm\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522028    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-home\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522134 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf5jb\" (UniqueName: \"kubernetes.io/projected/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-kube-api-access-qf5jb\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522161    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tls-certs\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522213 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522197    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tmp-dir\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522386 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522346    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-model-cache\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522477 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522455    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-home\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522535 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522487    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tmp-dir\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.522686 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.522666    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-model-cache\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.524376 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.524352    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-dshm\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.524551 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.524535    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tls-certs\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.530083 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.530065    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf5jb\" (UniqueName: \"kubernetes.io/projected/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-kube-api-access-qf5jb\") pod \"roll-v1-kserve-69cf8bcfc-lzpdf\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") " pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.665101 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.665080    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:00.932297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.932270    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"]
Sep 09 20:54:00.934598 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:54:00.934568    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9e9592_4cbe_4195_9aed_24ef0bca9c64.slice/crio-69b1b4b8adbfbf889022cddca3bec5add18e4571be84a5b07ddd00a66e0f1762 WatchSource:0}: Error finding container 69b1b4b8adbfbf889022cddca3bec5add18e4571be84a5b07ddd00a66e0f1762: Status 404 returned error can't find the container with id 69b1b4b8adbfbf889022cddca3bec5add18e4571be84a5b07ddd00a66e0f1762
Sep 09 20:54:00.966394 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:00.966370    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" event={"ID":"ad9e9592-4cbe-4195-9aed-24ef0bca9c64","Type":"ContainerStarted","Data":"69b1b4b8adbfbf889022cddca3bec5add18e4571be84a5b07ddd00a66e0f1762"}
Sep 09 20:54:01.972625 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:01.972594    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" event={"ID":"ad9e9592-4cbe-4195-9aed-24ef0bca9c64","Type":"ContainerStarted","Data":"45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359"}
Sep 09 20:54:01.989522 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:01.989443    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" podStartSLOduration=1.98942527 podStartE2EDuration="1.98942527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:01.988445897 +0000 UTC m=+2580.919529216" watchObservedRunningTime="2026-09-09 20:54:01.98942527 +0000 UTC m=+2580.920508589"
Sep 09 20:54:02.654848 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:02.654817    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:02.665650 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:02.665630    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:10.665948 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:10.665908    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:10.677814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:10.677789    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:10.838228 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:10.838197    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:11.020518 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:11.020444    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:54:29.644233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:29.644164    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:29.737956 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:29.737924    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:36.115726 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:36.115694    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:39.787915 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:39.787884    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:41.384793 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:41.384757    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:41.666937 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:41.666856    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:43.047880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:43.047842    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:43.148579 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:43.148541    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:45.988932 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:45.988894    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:46.090313 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:46.090269    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:55.932120 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:54:55.932085    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:01.929579 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:01.929544    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:26.155494 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.155458    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"]
Sep 09 20:55:26.158719 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.158704    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.160994 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.160974    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Sep 09 20:55:26.162019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.161998    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-crlf8\""
Sep 09 20:55:26.162019 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.162013    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Sep 09 20:55:26.162181 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.162109    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Sep 09 20:55:26.168126 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.168101    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"]
Sep 09 20:55:26.266815 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.266790    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:26.283951 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.283927    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.284042 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.283961    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh4pk\" (UniqueName: \"kubernetes.io/projected/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-kube-api-access-wh4pk\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.284085 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.284055    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.284127 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.284114    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.284165 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.284135    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.284202 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.284164    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.384829 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.384806    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.384929 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.384845    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.384929 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.384862    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.385012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.384983    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.385063 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.385024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.385063 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.385047    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh4pk\" (UniqueName: \"kubernetes.io/projected/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-kube-api-access-wh4pk\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.385276 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.385234    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.385276 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.385245    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.385411 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.385300    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.387171 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.387151    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.387481 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.387463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.393806 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.393782    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh4pk\" (UniqueName: \"kubernetes.io/projected/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-kube-api-access-wh4pk\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.469778 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.469713    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:26.472321 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.472299    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:26.740236 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:26.740187    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"]
Sep 09 20:55:26.742359 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:55:26.742333    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a608fa1_d7be_4213_ad25_3c0b41eb88e2.slice/crio-1c0d44b8856136d08c80e0c326f59259652bff2d328b5e59ef91ab1050a4dd18 WatchSource:0}: Error finding container 1c0d44b8856136d08c80e0c326f59259652bff2d328b5e59ef91ab1050a4dd18: Status 404 returned error can't find the container with id 1c0d44b8856136d08c80e0c326f59259652bff2d328b5e59ef91ab1050a4dd18
Sep 09 20:55:27.346459 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:27.346419    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" event={"ID":"8a608fa1-d7be-4213-ad25-3c0b41eb88e2","Type":"ContainerStarted","Data":"608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb"}
Sep 09 20:55:27.346827 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:27.346476    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" event={"ID":"8a608fa1-d7be-4213-ad25-3c0b41eb88e2","Type":"ContainerStarted","Data":"1c0d44b8856136d08c80e0c326f59259652bff2d328b5e59ef91ab1050a4dd18"}
Sep 09 20:55:27.364622 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:27.364324    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" podStartSLOduration=1.364306188 podStartE2EDuration="1.364306188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:27.36378565 +0000 UTC m=+2666.294868966" watchObservedRunningTime="2026-09-09 20:55:27.364306188 +0000 UTC m=+2666.295389506"
Sep 09 20:55:27.470759 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:27.470735    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:36.470164 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:36.470129    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:36.481977 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:36.481949    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:37.399574 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:37.399545    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:55:39.920650 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:39.920612    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:50.471869 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:50.471831    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:50.574475 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:50.574434    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:50.682233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:50.682207    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:50.956012 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:50.955978    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"]
Sep 09 20:55:51.000756 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:51.000728    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:52.173517 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:52.173486    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:55.543042 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:55:55.542961    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:01.675437 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:01.675414    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:56:01.679888 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:01.679863    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 20:56:01.687731 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:01.687709    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:56:01.692051 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:01.692033    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 20:56:05.974753 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:05.974705    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" podUID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" containerName="main" containerID="cri-o://45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359" gracePeriod=45
Sep 09 20:56:06.010096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.010062    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" podUID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Sep 09 20:56:06.213821 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.213798    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:56:06.316832 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.316770    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-dshm\") pod \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") "
Sep 09 20:56:06.316832 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.316821    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-home\") pod \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") "
Sep 09 20:56:06.317037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.316835    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-model-cache\") pod \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") "
Sep 09 20:56:06.317037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.316879    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf5jb\" (UniqueName: \"kubernetes.io/projected/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-kube-api-access-qf5jb\") pod \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") "
Sep 09 20:56:06.317037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.316897    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tls-certs\") pod \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") "
Sep 09 20:56:06.317037 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.316945    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tmp-dir\") pod \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\" (UID: \"ad9e9592-4cbe-4195-9aed-24ef0bca9c64\") "
Sep 09 20:56:06.317233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.317071    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-home" (OuterVolumeSpecName: "home") pod "ad9e9592-4cbe-4195-9aed-24ef0bca9c64" (UID: "ad9e9592-4cbe-4195-9aed-24ef0bca9c64"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:06.317233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.317093    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad9e9592-4cbe-4195-9aed-24ef0bca9c64" (UID: "ad9e9592-4cbe-4195-9aed-24ef0bca9c64"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:06.317362 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.317241    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad9e9592-4cbe-4195-9aed-24ef0bca9c64" (UID: "ad9e9592-4cbe-4195-9aed-24ef0bca9c64"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:06.319075 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.319049    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-dshm" (OuterVolumeSpecName: "dshm") pod "ad9e9592-4cbe-4195-9aed-24ef0bca9c64" (UID: "ad9e9592-4cbe-4195-9aed-24ef0bca9c64"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:06.319175 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.319117    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad9e9592-4cbe-4195-9aed-24ef0bca9c64" (UID: "ad9e9592-4cbe-4195-9aed-24ef0bca9c64"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:56:06.319175 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.319123    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-kube-api-access-qf5jb" (OuterVolumeSpecName: "kube-api-access-qf5jb") pod "ad9e9592-4cbe-4195-9aed-24ef0bca9c64" (UID: "ad9e9592-4cbe-4195-9aed-24ef0bca9c64"). InnerVolumeSpecName "kube-api-access-qf5jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:06.418417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.418391    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:56:06.418417 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.418412    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:56:06.418548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.418423    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf5jb\" (UniqueName: \"kubernetes.io/projected/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-kube-api-access-qf5jb\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:56:06.418548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.418432    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:56:06.418548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.418440    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:56:06.418548 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.418447    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad9e9592-4cbe-4195-9aed-24ef0bca9c64-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:56:06.524333 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.524295    2573 generic.go:356] "Generic (PLEG): container finished" podID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" containerID="45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359" exitCode=0
Sep 09 20:56:06.524511 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.524356    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"
Sep 09 20:56:06.524511 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.524383    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" event={"ID":"ad9e9592-4cbe-4195-9aed-24ef0bca9c64","Type":"ContainerDied","Data":"45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359"}
Sep 09 20:56:06.524511 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.524471    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf" event={"ID":"ad9e9592-4cbe-4195-9aed-24ef0bca9c64","Type":"ContainerDied","Data":"69b1b4b8adbfbf889022cddca3bec5add18e4571be84a5b07ddd00a66e0f1762"}
Sep 09 20:56:06.524511 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.524503    2573 scope.go:117] "RemoveContainer" containerID="45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359"
Sep 09 20:56:06.532752 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.532737    2573 scope.go:117] "RemoveContainer" containerID="45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359"
Sep 09 20:56:06.533007 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:56:06.532988    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359\": container with ID starting with 45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359 not found: ID does not exist" containerID="45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359"
Sep 09 20:56:06.533059 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.533015    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359"} err="failed to get container status \"45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359\": rpc error: code = NotFound desc = could not find container \"45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359\": container with ID starting with 45f21ca4b38123b95908e4ca5594880c033e57b266a696f7e820c13f46008359 not found: ID does not exist"
Sep 09 20:56:06.549032 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.549011    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"]
Sep 09 20:56:06.553427 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:06.553404    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rollback-b8fdb441/roll-v1-kserve-69cf8bcfc-lzpdf"]
Sep 09 20:56:07.641769 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:07.641741    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" path="/var/lib/kubelet/pods/ad9e9592-4cbe-4195-9aed-24ef0bca9c64/volumes"
Sep 09 20:56:26.676788 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:26.676753    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:32.899208 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:32.899166    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:38.742388 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:38.742353    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:38.814880 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:38.814823    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:40.544352 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.544315    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"]
Sep 09 20:56:40.544847 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.544828    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" containerName="main"
Sep 09 20:56:40.544965 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.544854    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" containerName="main"
Sep 09 20:56:40.544965 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.544944    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad9e9592-4cbe-4195-9aed-24ef0bca9c64" containerName="main"
Sep 09 20:56:40.547839 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.547817    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.550307 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.550286    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"default-dockercfg-tx7sq\""
Sep 09 20:56:40.551447 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.551422    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"kube-root-ca.crt\""
Sep 09 20:56:40.551601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.551431    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"openshift-service-ca.crt\""
Sep 09 20:56:40.551812 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.551442    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-force-stop-route-owner-5f630121\"/\"owner-v2-kserve-self-signed-certs\""
Sep 09 20:56:40.558728 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.558706    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"]
Sep 09 20:56:40.592151 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.592122    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:40.677969 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.677946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-home\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.678087 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.677978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-dshm\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.678087 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.678014    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfl9j\" (UniqueName: \"kubernetes.io/projected/c4296895-1315-4671-af64-8456d2a42fa7-kube-api-access-qfl9j\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.678169 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.678085    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.678169 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.678111    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4296895-1315-4671-af64-8456d2a42fa7-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.678169 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.678154    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-model-cache\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.778757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.778729    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-home\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.778757 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.778758    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-dshm\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.778935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.778789    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfl9j\" (UniqueName: \"kubernetes.io/projected/c4296895-1315-4671-af64-8456d2a42fa7-kube-api-access-qfl9j\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.778935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.778843    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.778935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.778865    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4296895-1315-4671-af64-8456d2a42fa7-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.778935 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.778919    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-model-cache\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.779172 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.779154    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-home\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.779222 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.779210    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-tmp-dir\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.779319 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.779302    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-model-cache\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.781051 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.781034    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-dshm\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.781453 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.781431    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4296895-1315-4671-af64-8456d2a42fa7-tls-certs\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.790126 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.790101    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfl9j\" (UniqueName: \"kubernetes.io/projected/c4296895-1315-4671-af64-8456d2a42fa7-kube-api-access-qfl9j\") pod \"owner-v2-kserve-5cf8959f95-llkq5\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") " pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:40.858762 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:40.858735    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:41.131938 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:41.131916    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"]
Sep 09 20:56:41.133440 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:56:41.133416    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4296895_1315_4671_af64_8456d2a42fa7.slice/crio-ccd7585113a12d0ee8e915d00bd97d037c56d762204d12f23c430b64b6027cfa WatchSource:0}: Error finding container ccd7585113a12d0ee8e915d00bd97d037c56d762204d12f23c430b64b6027cfa: Status 404 returned error can't find the container with id ccd7585113a12d0ee8e915d00bd97d037c56d762204d12f23c430b64b6027cfa
Sep 09 20:56:41.684492 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:41.684461    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5" event={"ID":"c4296895-1315-4671-af64-8456d2a42fa7","Type":"ContainerStarted","Data":"b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19"}
Sep 09 20:56:41.684875 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:41.684507    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5" event={"ID":"c4296895-1315-4671-af64-8456d2a42fa7","Type":"ContainerStarted","Data":"ccd7585113a12d0ee8e915d00bd97d037c56d762204d12f23c430b64b6027cfa"}
Sep 09 20:56:41.705429 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:41.705386    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5" podStartSLOduration=1.705372189 podStartE2EDuration="1.705372189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:41.703616597 +0000 UTC m=+2740.634699918" watchObservedRunningTime="2026-09-09 20:56:41.705372189 +0000 UTC m=+2740.636455506"
Sep 09 20:56:41.859526 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:41.859495    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:50.859332 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:50.859301    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:50.871628 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:50.871604    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:56:51.738374 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:56:51.738342    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:57:12.553832 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.553780    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:12.558878 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.558848    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"]
Sep 09 20:57:12.567858 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:57:12.567828    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 20:57:12.567858 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:57:12.567858 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:57:12.567858 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb" cmd=["/bin/sleep","15"]
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.567872    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" message="" err=<
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:57:12.567884    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" podUID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" containerName="main" containerID="cri-o://608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb"
Sep 09 20:57:12.568096 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.567929    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" podUID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" containerName="main" containerID="cri-o://608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb" gracePeriod=60
Sep 09 20:57:12.825814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.825737    2573 generic.go:356] "Generic (PLEG): container finished" podID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" containerID="608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb" exitCode=0
Sep 09 20:57:12.825814 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.825772    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" event={"ID":"8a608fa1-d7be-4213-ad25-3c0b41eb88e2","Type":"ContainerDied","Data":"608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb"}
Sep 09 20:57:12.911570 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.911548    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:57:12.969783 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:12.969729    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:13.025044 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025022    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tmp-dir\") pod \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") "
Sep 09 20:57:13.025152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025057    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-dshm\") pod \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") "
Sep 09 20:57:13.025152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025081    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh4pk\" (UniqueName: \"kubernetes.io/projected/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-kube-api-access-wh4pk\") pod \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") "
Sep 09 20:57:13.025152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025098    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-model-cache\") pod \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") "
Sep 09 20:57:13.025152 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025117    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-home\") pod \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") "
Sep 09 20:57:13.025450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025168    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tls-certs\") pod \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\" (UID: \"8a608fa1-d7be-4213-ad25-3c0b41eb88e2\") "
Sep 09 20:57:13.025450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025302    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a608fa1-d7be-4213-ad25-3c0b41eb88e2" (UID: "8a608fa1-d7be-4213-ad25-3c0b41eb88e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:13.025450 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025381    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a608fa1-d7be-4213-ad25-3c0b41eb88e2" (UID: "8a608fa1-d7be-4213-ad25-3c0b41eb88e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:13.025559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025455    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-home" (OuterVolumeSpecName: "home") pod "8a608fa1-d7be-4213-ad25-3c0b41eb88e2" (UID: "8a608fa1-d7be-4213-ad25-3c0b41eb88e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:13.025559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025503    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:57:13.025559 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.025522    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:57:13.027371 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.027341    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a608fa1-d7be-4213-ad25-3c0b41eb88e2" (UID: "8a608fa1-d7be-4213-ad25-3c0b41eb88e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:57:13.027371 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.027363    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-kube-api-access-wh4pk" (OuterVolumeSpecName: "kube-api-access-wh4pk") pod "8a608fa1-d7be-4213-ad25-3c0b41eb88e2" (UID: "8a608fa1-d7be-4213-ad25-3c0b41eb88e2"). InnerVolumeSpecName "kube-api-access-wh4pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:13.027503 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.027380    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-dshm" (OuterVolumeSpecName: "dshm") pod "8a608fa1-d7be-4213-ad25-3c0b41eb88e2" (UID: "8a608fa1-d7be-4213-ad25-3c0b41eb88e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:13.126620 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.126596    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:57:13.126620 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.126617    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:57:13.126764 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.126627    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh4pk\" (UniqueName: \"kubernetes.io/projected/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-kube-api-access-wh4pk\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:57:13.126764 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.126635    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a608fa1-d7be-4213-ad25-3c0b41eb88e2-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:57:13.830970 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.830936    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6" event={"ID":"8a608fa1-d7be-4213-ad25-3c0b41eb88e2","Type":"ContainerDied","Data":"1c0d44b8856136d08c80e0c326f59259652bff2d328b5e59ef91ab1050a4dd18"}
Sep 09 20:57:13.830970 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.830953    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"
Sep 09 20:57:13.831458 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.830991    2573 scope.go:117] "RemoveContainer" containerID="608cb8ce301e32bcaab46e3aa8cca2f78e3e611709262983c537570955e12cbb"
Sep 09 20:57:13.847945 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.847922    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"]
Sep 09 20:57:13.852038 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:13.852018    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995q42l6"]
Sep 09 20:57:15.641666 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:15.641637    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" path="/var/lib/kubelet/pods/8a608fa1-d7be-4213-ad25-3c0b41eb88e2/volumes"
Sep 09 20:57:27.556084 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:27.556007    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:47.919833 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:47.919795    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:48.040827 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.040790    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:48.172165 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.172083    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"]
Sep 09 20:57:48.172460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.172444    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" containerName="main"
Sep 09 20:57:48.172460 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.172459    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" containerName="main"
Sep 09 20:57:48.172606 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.172513    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a608fa1-d7be-4213-ad25-3c0b41eb88e2" containerName="main"
Sep 09 20:57:48.175416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.175393    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.178048 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.178026    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Sep 09 20:57:48.178191 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.178166    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"router-gateway-1-openshift-default-dockercfg-mrnq7\""
Sep 09 20:57:48.178282 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.178091    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-gw-ca-root-cert\""
Sep 09 20:57:48.178350 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.178111    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Sep 09 20:57:48.194200 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.194175    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"]
Sep 09 20:57:48.226330 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.226308    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"]
Sep 09 20:57:48.250057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.250033    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:48.288187 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288161    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-certs\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-credential-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288214    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xngn9\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-kube-api-access-xngn9\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288277    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-data\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288410 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288326    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-token\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288410 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288352    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288410 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288400    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3dd029f-f42a-4796-a085-40fd1907f86e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288451    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-envoy\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.288516 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.288479    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-podinfo\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.357962 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.357933    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:48.389201 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389175    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-certs\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389303 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389204    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-credential-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389303 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389225    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xngn9\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-kube-api-access-xngn9\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389303 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389246    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-data\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389439 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389306    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-token\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389439 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389336    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389439 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389363    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3dd029f-f42a-4796-a085-40fd1907f86e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389439 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389405    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-envoy\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389645 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389441    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-podinfo\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389755 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389730    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-certs\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389850 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389767    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-data\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389850 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389816    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.389943 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.389862    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-credential-socket\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.390101 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.390082    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3dd029f-f42a-4796-a085-40fd1907f86e-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.391591 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.391571    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-envoy\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.391831 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.391813    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-podinfo\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.401776 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.401756    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-token\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.402196 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.402173    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xngn9\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-kube-api-access-xngn9\") pod \"router-gateway-1-openshift-default-5bb8f7bc87-mcsvh\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.489205 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.489148    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:48.761301 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.761276    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"]
Sep 09 20:57:48.763611 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:57:48.763586    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3dd029f_f42a_4796_a085_40fd1907f86e.slice/crio-67ac1eb1f4bd6d4a64c6f0cebe222f54281982394d0b1eb81635041ec60482f8 WatchSource:0}: Error finding container 67ac1eb1f4bd6d4a64c6f0cebe222f54281982394d0b1eb81635041ec60482f8: Status 404 returned error can't find the container with id 67ac1eb1f4bd6d4a64c6f0cebe222f54281982394d0b1eb81635041ec60482f8
Sep 09 20:57:48.765489 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.765451    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:57:48.765587 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.765535    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:57:48.765587 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.765569    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:57:48.989576 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.989545    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" event={"ID":"d3dd029f-f42a-4796-a085-40fd1907f86e","Type":"ContainerStarted","Data":"c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"}
Sep 09 20:57:48.989926 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:48.989595    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" event={"ID":"d3dd029f-f42a-4796-a085-40fd1907f86e","Type":"ContainerStarted","Data":"67ac1eb1f4bd6d4a64c6f0cebe222f54281982394d0b1eb81635041ec60482f8"}
Sep 09 20:57:49.011856 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:49.011767    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podStartSLOduration=1.01175284 podStartE2EDuration="1.01175284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:49.00980089 +0000 UTC m=+2807.940884212" watchObservedRunningTime="2026-09-09 20:57:49.01175284 +0000 UTC m=+2807.942836157"
Sep 09 20:57:49.489632 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:49.489588    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:49.494648 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:49.494623    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:49.994110 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:49.994081    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:49.994917 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:49.994898    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:57:51.887805 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:51.887778    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:52.927744 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:52.927711    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:58.444554 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.444519    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"]
Sep 09 20:57:58.448067 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.448041    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.451380 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.451357    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Sep 09 20:57:58.451502 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.451361    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"default-dockercfg-8z5sz\""
Sep 09 20:57:58.457576 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.457550    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"]
Sep 09 20:57:58.482487 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.482461    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:58.572793 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.572769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.572913 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.572880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.572984 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.572910    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.572984 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.572944    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8cdh\" (UniqueName: \"kubernetes.io/projected/3a0a16a3-b9f8-47ff-acae-0bd71e795823-kube-api-access-h8cdh\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.573087 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.572988    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.573087 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.573011    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.673770 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.673745    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.673879 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.673778    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.673879 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.673802    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8cdh\" (UniqueName: \"kubernetes.io/projected/3a0a16a3-b9f8-47ff-acae-0bd71e795823-kube-api-access-h8cdh\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.673999 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.673915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.673999 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.673938    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.673999 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.673968    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.674266 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.674225    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.674364 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.674302    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.674364 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.674340    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.675933 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.675914    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.676306 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.676289    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.683699 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.683678    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8cdh\" (UniqueName: \"kubernetes.io/projected/3a0a16a3-b9f8-47ff-acae-0bd71e795823-kube-api-access-h8cdh\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:58.759385 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:58.759335    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:57:59.031696 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:57:59.031642    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"]
Sep 09 20:57:59.033905 ip-10-0-132-68 kubenswrapper[2573]: W0909 20:57:59.033879    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0a16a3_b9f8_47ff_acae_0bd71e795823.slice/crio-2d1d1791739b0e1a40b5c9158b08e00080afcc2972a9e3a5fc5004131b1ae145 WatchSource:0}: Error finding container 2d1d1791739b0e1a40b5c9158b08e00080afcc2972a9e3a5fc5004131b1ae145: Status 404 returned error can't find the container with id 2d1d1791739b0e1a40b5c9158b08e00080afcc2972a9e3a5fc5004131b1ae145
Sep 09 20:58:00.039861 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:00.039825    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" event={"ID":"3a0a16a3-b9f8-47ff-acae-0bd71e795823","Type":"ContainerStarted","Data":"2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"}
Sep 09 20:58:00.040377 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:00.039881    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" event={"ID":"3a0a16a3-b9f8-47ff-acae-0bd71e795823","Type":"ContainerStarted","Data":"2d1d1791739b0e1a40b5c9158b08e00080afcc2972a9e3a5fc5004131b1ae145"}
Sep 09 20:58:00.062053 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:00.061977    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" podStartSLOduration=2.061954749 podStartE2EDuration="2.061954749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:00.060105804 +0000 UTC m=+2818.991189123" watchObservedRunningTime="2026-09-09 20:58:00.061954749 +0000 UTC m=+2818.993038068"
Sep 09 20:58:00.216968 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:00.216938    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:00.310306 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:00.310187    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:00.759473 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:00.759439    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:58:03.244586 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.244546    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5" podUID="c4296895-1315-4671-af64-8456d2a42fa7" containerName="main" containerID="cri-o://b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19" gracePeriod=45
Sep 09 20:58:03.519849 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.519823    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:58:03.617233 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617200    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-dshm\") pod \"c4296895-1315-4671-af64-8456d2a42fa7\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") "
Sep 09 20:58:03.617416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617278    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-home\") pod \"c4296895-1315-4671-af64-8456d2a42fa7\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") "
Sep 09 20:58:03.617416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617337    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfl9j\" (UniqueName: \"kubernetes.io/projected/c4296895-1315-4671-af64-8456d2a42fa7-kube-api-access-qfl9j\") pod \"c4296895-1315-4671-af64-8456d2a42fa7\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") "
Sep 09 20:58:03.617416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617362    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-tmp-dir\") pod \"c4296895-1315-4671-af64-8456d2a42fa7\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") "
Sep 09 20:58:03.617416 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617398    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4296895-1315-4671-af64-8456d2a42fa7-tls-certs\") pod \"c4296895-1315-4671-af64-8456d2a42fa7\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") "
Sep 09 20:58:03.617633 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617429    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-model-cache\") pod \"c4296895-1315-4671-af64-8456d2a42fa7\" (UID: \"c4296895-1315-4671-af64-8456d2a42fa7\") "
Sep 09 20:58:03.617633 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617471    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-home" (OuterVolumeSpecName: "home") pod "c4296895-1315-4671-af64-8456d2a42fa7" (UID: "c4296895-1315-4671-af64-8456d2a42fa7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:58:03.617633 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617614    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4296895-1315-4671-af64-8456d2a42fa7" (UID: "c4296895-1315-4671-af64-8456d2a42fa7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:58:03.617781 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617716    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4296895-1315-4671-af64-8456d2a42fa7" (UID: "c4296895-1315-4671-af64-8456d2a42fa7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:58:03.617816 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617777    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:58:03.617816 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.617794    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:58:03.619655 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.619630    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4296895-1315-4671-af64-8456d2a42fa7-kube-api-access-qfl9j" (OuterVolumeSpecName: "kube-api-access-qfl9j") pod "c4296895-1315-4671-af64-8456d2a42fa7" (UID: "c4296895-1315-4671-af64-8456d2a42fa7"). InnerVolumeSpecName "kube-api-access-qfl9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:58:03.619762 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.619653    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4296895-1315-4671-af64-8456d2a42fa7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4296895-1315-4671-af64-8456d2a42fa7" (UID: "c4296895-1315-4671-af64-8456d2a42fa7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:58:03.619762 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.619700    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-dshm" (OuterVolumeSpecName: "dshm") pod "c4296895-1315-4671-af64-8456d2a42fa7" (UID: "c4296895-1315-4671-af64-8456d2a42fa7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:58:03.718813 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.718783    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfl9j\" (UniqueName: \"kubernetes.io/projected/c4296895-1315-4671-af64-8456d2a42fa7-kube-api-access-qfl9j\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:58:03.718813 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.718808    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4296895-1315-4671-af64-8456d2a42fa7-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:58:03.718813 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.718819    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:58:03.719017 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:03.718827    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4296895-1315-4671-af64-8456d2a42fa7-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 20:58:04.067288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.065108    2573 generic.go:356] "Generic (PLEG): container finished" podID="c4296895-1315-4671-af64-8456d2a42fa7" containerID="b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19" exitCode=0
Sep 09 20:58:04.067288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.065156    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5" event={"ID":"c4296895-1315-4671-af64-8456d2a42fa7","Type":"ContainerDied","Data":"b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19"}
Sep 09 20:58:04.067288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.065216    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5" event={"ID":"c4296895-1315-4671-af64-8456d2a42fa7","Type":"ContainerDied","Data":"ccd7585113a12d0ee8e915d00bd97d037c56d762204d12f23c430b64b6027cfa"}
Sep 09 20:58:04.067288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.065262    2573 scope.go:117] "RemoveContainer" containerID="b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19"
Sep 09 20:58:04.067288 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.065434    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"
Sep 09 20:58:04.081811 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.081789    2573 scope.go:117] "RemoveContainer" containerID="b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19"
Sep 09 20:58:04.082085 ip-10-0-132-68 kubenswrapper[2573]: E0909 20:58:04.082064    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19\": container with ID starting with b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19 not found: ID does not exist" containerID="b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19"
Sep 09 20:58:04.082133 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.082092    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19"} err="failed to get container status \"b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19\": rpc error: code = NotFound desc = could not find container \"b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19\": container with ID starting with b64ecb510da5b6c58d63906791944a619e93bf1b1a8f7295d64c26804512fd19 not found: ID does not exist"
Sep 09 20:58:04.091022 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.090996    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"]
Sep 09 20:58:04.095928 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:04.095902    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-force-stop-route-owner-5f630121/owner-v2-kserve-5cf8959f95-llkq5"]
Sep 09 20:58:05.642048 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:05.642009    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4296895-1315-4671-af64-8456d2a42fa7" path="/var/lib/kubelet/pods/c4296895-1315-4671-af64-8456d2a42fa7/volumes"
Sep 09 20:58:06.036475 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:06.036374    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 20:58:06.420664 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:06.420632    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:08.089313 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.089278    2573 generic.go:356] "Generic (PLEG): container finished" podID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerID="c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425" exitCode=0
Sep 09 20:58:08.089771 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.089319    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" event={"ID":"d3dd029f-f42a-4796-a085-40fd1907f86e","Type":"ContainerDied","Data":"c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"}
Sep 09 20:58:08.089771 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.089645    2573 scope.go:117] "RemoveContainer" containerID="c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"
Sep 09 20:58:08.090520 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.090494    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:58:08.090601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.090559    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:58:08.090601 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.090591    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 20:58:08.761111 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.761071    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:58:08.780463 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:08.780431    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:58:09.095187 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:09.095097    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" event={"ID":"d3dd029f-f42a-4796-a085-40fd1907f86e","Type":"ContainerStarted","Data":"428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a"}
Sep 09 20:58:09.105980 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:09.105957    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 20:58:09.490024 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:09.489984    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:58:09.490215 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:09.490165    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 20:58:10.489999 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:10.489959    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 20:58:11.490214 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:11.490171    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 20:58:12.490361 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:12.490313    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 20:58:13.490080 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:13.490037    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 20:58:14.493303 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:14.493271    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:58:14.493714 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:14.493605    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:58:14.494358 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:14.494337    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 20:58:30.490867 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:30.490833    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:36.676297 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:36.676238    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:36.796057 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:36.796024    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:49.940227 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:49.940194    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:50.038567 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:50.038542    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:54.515525 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:54.515431    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:54.729557 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:54.729523    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:57.935674 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:58:57.935630    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:04.092905 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:59:04.092870    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:05.634284 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:59:05.634231    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:34.758469 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:59:34.758411    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:35.036188 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:59:35.036106    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:50.316569 ip-10-0-132-68 kubenswrapper[2573]: I0909 20:59:50.316521    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:02.444947 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:02.444909    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:09.790113 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:09.790080    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:09.886881 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:09.886847    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:14.332412 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.332379    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"]
Sep 09 21:00:14.340816 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:00:14.340759    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 21:00:14.340816 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 21:00:14.340816 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 21:00:14.340816 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766" cmd=["/bin/sleep","15"]
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.340829    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" message="" err=<
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:00:14.340843    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" podUID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" containerName="main" containerID="cri-o://2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"
Sep 09 21:00:14.341106 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.340887    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" podUID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" containerName="main" containerID="cri-o://2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766" gracePeriod=60
Sep 09 21:00:14.357938 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.357909    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:14.591945 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.591892    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 21:00:14.656014 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.655985    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" containerID="2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766" exitCode=0
Sep 09 21:00:14.656133 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.656033    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"
Sep 09 21:00:14.656133 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.656074    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" event={"ID":"3a0a16a3-b9f8-47ff-acae-0bd71e795823","Type":"ContainerDied","Data":"2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"}
Sep 09 21:00:14.656133 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.656128    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b" event={"ID":"3a0a16a3-b9f8-47ff-acae-0bd71e795823","Type":"ContainerDied","Data":"2d1d1791739b0e1a40b5c9158b08e00080afcc2972a9e3a5fc5004131b1ae145"}
Sep 09 21:00:14.656272 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.656149    2573 scope.go:117] "RemoveContainer" containerID="2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"
Sep 09 21:00:14.664536 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.664513    2573 scope.go:117] "RemoveContainer" containerID="2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"
Sep 09 21:00:14.664814 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:00:14.664796    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766\": container with ID starting with 2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766 not found: ID does not exist" containerID="2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"
Sep 09 21:00:14.664868 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.664822    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766"} err="failed to get container status \"2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766\": rpc error: code = NotFound desc = could not find container \"2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766\": container with ID starting with 2be8c5a9700f9deb23b77c4ec7a3aa376a56884216595518cce26b42978ee766 not found: ID does not exist"
Sep 09 21:00:14.749293 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749269    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8cdh\" (UniqueName: \"kubernetes.io/projected/3a0a16a3-b9f8-47ff-acae-0bd71e795823-kube-api-access-h8cdh\") pod \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") "
Sep 09 21:00:14.749392 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749369    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tmp-dir\") pod \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") "
Sep 09 21:00:14.749446 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749395    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-dshm\") pod \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") "
Sep 09 21:00:14.749446 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749412    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-home\") pod \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") "
Sep 09 21:00:14.749446 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749429    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tls-certs\") pod \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") "
Sep 09 21:00:14.749446 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749444    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-model-cache\") pod \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\" (UID: \"3a0a16a3-b9f8-47ff-acae-0bd71e795823\") "
Sep 09 21:00:14.749686 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749661    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a0a16a3-b9f8-47ff-acae-0bd71e795823" (UID: "3a0a16a3-b9f8-47ff-acae-0bd71e795823"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:14.749750 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749669    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-home" (OuterVolumeSpecName: "home") pod "3a0a16a3-b9f8-47ff-acae-0bd71e795823" (UID: "3a0a16a3-b9f8-47ff-acae-0bd71e795823"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:14.749810 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.749772    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a0a16a3-b9f8-47ff-acae-0bd71e795823" (UID: "3a0a16a3-b9f8-47ff-acae-0bd71e795823"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:14.751404 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.751375    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a0a16a3-b9f8-47ff-acae-0bd71e795823-kube-api-access-h8cdh" (OuterVolumeSpecName: "kube-api-access-h8cdh") pod "3a0a16a3-b9f8-47ff-acae-0bd71e795823" (UID: "3a0a16a3-b9f8-47ff-acae-0bd71e795823"). InnerVolumeSpecName "kube-api-access-h8cdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:14.751483 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.751418    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a0a16a3-b9f8-47ff-acae-0bd71e795823" (UID: "3a0a16a3-b9f8-47ff-acae-0bd71e795823"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:00:14.751483 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.751454    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-dshm" (OuterVolumeSpecName: "dshm") pod "3a0a16a3-b9f8-47ff-acae-0bd71e795823" (UID: "3a0a16a3-b9f8-47ff-acae-0bd71e795823"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:14.850474 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.850417    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:14.850474 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.850439    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:14.850474 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.850450    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:14.850474 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.850460    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a0a16a3-b9f8-47ff-acae-0bd71e795823-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:14.850474 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.850473    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a0a16a3-b9f8-47ff-acae-0bd71e795823-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:14.850728 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.850485    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8cdh\" (UniqueName: \"kubernetes.io/projected/3a0a16a3-b9f8-47ff-acae-0bd71e795823-kube-api-access-h8cdh\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:14.982798 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.982771    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"]
Sep 09 21:00:14.987322 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:14.987302    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff5899785f4b"]
Sep 09 21:00:15.642086 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:15.642055    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" path="/var/lib/kubelet/pods/3a0a16a3-b9f8-47ff-acae-0bd71e795823/volumes"
Sep 09 21:00:19.298543 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:19.298513    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:19.870702 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:19.870668    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:24.248136 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:24.248057    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"]
Sep 09 21:00:24.248548 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:24.248274    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy" containerID="cri-o://428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a" gracePeriod=30
Sep 09 21:00:24.350581 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:24.350547    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:27.607409 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.607363    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:27.878867 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.878781    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"]
Sep 09 21:00:27.879196 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.879178    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4296895-1315-4671-af64-8456d2a42fa7" containerName="main"
Sep 09 21:00:27.879304 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.879199    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4296895-1315-4671-af64-8456d2a42fa7" containerName="main"
Sep 09 21:00:27.879304 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.879212    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" containerName="main"
Sep 09 21:00:27.879304 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.879220    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" containerName="main"
Sep 09 21:00:27.879460 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.879446    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0a16a3-b9f8-47ff-acae-0bd71e795823" containerName="main"
Sep 09 21:00:27.879509 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.879466    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4296895-1315-4671-af64-8456d2a42fa7" containerName="main"
Sep 09 21:00:27.882169 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.882150    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:27.884670 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.884646    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Sep 09 21:00:27.885709 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.885684    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-tjxwg\""
Sep 09 21:00:27.885814 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.885709    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-p25sj\""
Sep 09 21:00:27.885814 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.885786    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Sep 09 21:00:27.885814 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.885806    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Sep 09 21:00:27.903493 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.903469    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"]
Sep 09 21:00:27.919596 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:27.919546    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:28.051087 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.051057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc8nf\" (UniqueName: \"kubernetes.io/projected/0f27343a-c682-4725-86ef-4e07a1f98964-kube-api-access-jc8nf\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.051215 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.051101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f27343a-c682-4725-86ef-4e07a1f98964-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.152016 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.151963    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc8nf\" (UniqueName: \"kubernetes.io/projected/0f27343a-c682-4725-86ef-4e07a1f98964-kube-api-access-jc8nf\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.152016 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.152003    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f27343a-c682-4725-86ef-4e07a1f98964-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.154517 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.154496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f27343a-c682-4725-86ef-4e07a1f98964-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.160910 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.160888    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc8nf\" (UniqueName: \"kubernetes.io/projected/0f27343a-c682-4725-86ef-4e07a1f98964-kube-api-access-jc8nf\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.191582 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.191560    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.464660 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.464636    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"]
Sep 09 21:00:28.466929 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:00:28.466900    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f27343a_c682_4725_86ef_4e07a1f98964.slice/crio-6d30e74bbfb2d23df2eddbd8851547152aa27538bffd0b6952361354cc551463 WatchSource:0}: Error finding container 6d30e74bbfb2d23df2eddbd8851547152aa27538bffd0b6952361354cc551463: Status 404 returned error can't find the container with id 6d30e74bbfb2d23df2eddbd8851547152aa27538bffd0b6952361354cc551463
Sep 09 21:00:28.718246 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.718161    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" event={"ID":"0f27343a-c682-4725-86ef-4e07a1f98964","Type":"ContainerStarted","Data":"d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf"}
Sep 09 21:00:28.718246 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.718213    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" event={"ID":"0f27343a-c682-4725-86ef-4e07a1f98964","Type":"ContainerStarted","Data":"6d30e74bbfb2d23df2eddbd8851547152aa27538bffd0b6952361354cc551463"}
Sep 09 21:00:28.718738 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.718290    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:00:28.739451 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:28.739408    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" podStartSLOduration=1.739394981 podStartE2EDuration="1.739394981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:28.73799354 +0000 UTC m=+2967.669076858" watchObservedRunningTime="2026-09-09 21:00:28.739394981 +0000 UTC m=+2967.670478298"
Sep 09 21:00:29.491648 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.491625    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 21:00:29.564958 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.564930    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3dd029f-f42a-4796-a085-40fd1907f86e-istiod-ca-cert\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.565088 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.564985    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-podinfo\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.565088 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565006    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xngn9\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-kube-api-access-xngn9\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.565395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565185    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-envoy\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.565395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565280    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-token\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.565395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565315    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-data\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.565395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565342    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3dd029f-f42a-4796-a085-40fd1907f86e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:29.565395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565369    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-certs\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565408    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-credential-socket\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565431    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-socket\") pod \"d3dd029f-f42a-4796-a085-40fd1907f86e\" (UID: \"d3dd029f-f42a-4796-a085-40fd1907f86e\") "
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565614    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565633    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565703    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565776    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565827    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-data\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565845    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565859    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-credential-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.566194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.565873    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3dd029f-f42a-4796-a085-40fd1907f86e-istiod-ca-cert\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.567655 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.567615    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:29.567760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.567615    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-kube-api-access-xngn9" (OuterVolumeSpecName: "kube-api-access-xngn9") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "kube-api-access-xngn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:29.567760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.567653    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:00:29.567760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.567686    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3dd029f-f42a-4796-a085-40fd1907f86e" (UID: "d3dd029f-f42a-4796-a085-40fd1907f86e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:29.666994 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.666970    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-podinfo\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.666994 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.666990    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xngn9\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-kube-api-access-xngn9\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.667115 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.667000    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-envoy\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.667115 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.667008    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3dd029f-f42a-4796-a085-40fd1907f86e-istio-token\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.667115 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.667015    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3dd029f-f42a-4796-a085-40fd1907f86e-workload-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:00:29.728114 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.728084    2573 generic.go:356] "Generic (PLEG): container finished" podID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerID="428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a" exitCode=0
Sep 09 21:00:29.728480 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.728136    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"
Sep 09 21:00:29.728480 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.728174    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" event={"ID":"d3dd029f-f42a-4796-a085-40fd1907f86e","Type":"ContainerDied","Data":"428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a"}
Sep 09 21:00:29.728480 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.728278    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh" event={"ID":"d3dd029f-f42a-4796-a085-40fd1907f86e","Type":"ContainerDied","Data":"67ac1eb1f4bd6d4a64c6f0cebe222f54281982394d0b1eb81635041ec60482f8"}
Sep 09 21:00:29.728480 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.728309    2573 scope.go:117] "RemoveContainer" containerID="428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a"
Sep 09 21:00:29.737375 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.737280    2573 scope.go:117] "RemoveContainer" containerID="c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"
Sep 09 21:00:29.745656 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.745620    2573 scope.go:117] "RemoveContainer" containerID="428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a"
Sep 09 21:00:29.745922 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:00:29.745902    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a\": container with ID starting with 428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a not found: ID does not exist" containerID="428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a"
Sep 09 21:00:29.746013 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.745932    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a"} err="failed to get container status \"428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a\": rpc error: code = NotFound desc = could not find container \"428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a\": container with ID starting with 428f53dcf4fe6ce8f3eed1e2ffe04f70be3df5c9c55fc345cb80de6463b4749a not found: ID does not exist"
Sep 09 21:00:29.746013 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.745953    2573 scope.go:117] "RemoveContainer" containerID="c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"
Sep 09 21:00:29.746193 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:00:29.746177    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425\": container with ID starting with c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425 not found: ID does not exist" containerID="c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"
Sep 09 21:00:29.746269 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.746200    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425"} err="failed to get container status \"c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425\": rpc error: code = NotFound desc = could not find container \"c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425\": container with ID starting with c394de8caea6cda50fbc66e61ba1c6db4658dcf1cca9adc0439a06d0ac66b425 not found: ID does not exist"
Sep 09 21:00:29.747164 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.747145    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"]
Sep 09 21:00:29.751276 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:29.751237    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-5bb8f7bc87-mcsvh"]
Sep 09 21:00:31.642225 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:31.642192    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" path="/var/lib/kubelet/pods/d3dd029f-f42a-4796-a085-40fd1907f86e/volumes"
Sep 09 21:00:37.763649 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:37.763615    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:37.873463 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:37.873433    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:38.988429 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:38.988394    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:45.837722 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:45.837689    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:46.996662 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:46.996632    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:52.927316 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:52.927280    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:59.730378 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:00:59.730346    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:01:00.129402 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:00.129368    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:00.194442 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:00.194419    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:01.702145 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:01.702117    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:01:01.708274 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:01.708237    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:01:01.713708 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:01.713686    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:01:01.719289 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:01.719271    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:01:07.324336 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:07.324303    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:07.607368 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:07.607335    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:07.707503 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:07.707475    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:08.463874 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:08.463840    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:08.525460 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:08.525435    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:49.451955 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:49.451905    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:49.549343 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:49.549312    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:56.577371 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:01:56.577273    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:02.806579 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:02.806544    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:02.917762 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:02.917730    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:06.616815 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:06.616772    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:06.963098 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:06.963011    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:22.232827 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:22.232778    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:32.596728 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:32.596694    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:32.597859 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:32.597832    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"]
Sep 09 21:02:32.689197 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:32.689163    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:39.730192 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:39.730156    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:02:43.778820 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:43.778790    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:47.620219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:47.620181    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main" containerID="cri-o://d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf" gracePeriod=45
Sep 09 21:02:49.730699 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:49.730659    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:02:59.730560 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:59.730526    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:02:59.730988 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:02:59.730617    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:03:09.729940 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:09.729900    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:03:17.867540 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:17.867510    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:03:17.936016 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:17.935954    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f27343a-c682-4725-86ef-4e07a1f98964-tls-certs\") pod \"0f27343a-c682-4725-86ef-4e07a1f98964\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") "
Sep 09 21:03:17.936016 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:17.936007    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc8nf\" (UniqueName: \"kubernetes.io/projected/0f27343a-c682-4725-86ef-4e07a1f98964-kube-api-access-jc8nf\") pod \"0f27343a-c682-4725-86ef-4e07a1f98964\" (UID: \"0f27343a-c682-4725-86ef-4e07a1f98964\") "
Sep 09 21:03:17.938028 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:17.938003    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f27343a-c682-4725-86ef-4e07a1f98964-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0f27343a-c682-4725-86ef-4e07a1f98964" (UID: "0f27343a-c682-4725-86ef-4e07a1f98964"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:17.938135 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:17.938069    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f27343a-c682-4725-86ef-4e07a1f98964-kube-api-access-jc8nf" (OuterVolumeSpecName: "kube-api-access-jc8nf") pod "0f27343a-c682-4725-86ef-4e07a1f98964" (UID: "0f27343a-c682-4725-86ef-4e07a1f98964"). InnerVolumeSpecName "kube-api-access-jc8nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:18.036693 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.036668    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc8nf\" (UniqueName: \"kubernetes.io/projected/0f27343a-c682-4725-86ef-4e07a1f98964-kube-api-access-jc8nf\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:03:18.036693 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.036690    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f27343a-c682-4725-86ef-4e07a1f98964-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:03:18.466854 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.466822    2573 generic.go:356] "Generic (PLEG): container finished" podID="0f27343a-c682-4725-86ef-4e07a1f98964" containerID="d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf" exitCode=0
Sep 09 21:03:18.467009 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.466867    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"
Sep 09 21:03:18.467009 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.466909    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" event={"ID":"0f27343a-c682-4725-86ef-4e07a1f98964","Type":"ContainerDied","Data":"d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf"}
Sep 09 21:03:18.467009 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.466987    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l" event={"ID":"0f27343a-c682-4725-86ef-4e07a1f98964","Type":"ContainerDied","Data":"6d30e74bbfb2d23df2eddbd8851547152aa27538bffd0b6952361354cc551463"}
Sep 09 21:03:18.467118 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.467018    2573 scope.go:117] "RemoveContainer" containerID="d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf"
Sep 09 21:03:18.475627 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.475608    2573 scope.go:117] "RemoveContainer" containerID="d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf"
Sep 09 21:03:18.475902 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:03:18.475884    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf\": container with ID starting with d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf not found: ID does not exist" containerID="d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf"
Sep 09 21:03:18.475964 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.475911    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf"} err="failed to get container status \"d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf\": rpc error: code = NotFound desc = could not find container \"d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf\": container with ID starting with d085b63559cff685fc23072cb51792a6487f92f4fcdaf267d2d601bbebb5bacf not found: ID does not exist"
Sep 09 21:03:18.491886 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.491862    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"]
Sep 09 21:03:18.498503 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:18.498481    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schenxj2l"]
Sep 09 21:03:19.641225 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:19.641193    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" path="/var/lib/kubelet/pods/0f27343a-c682-4725-86ef-4e07a1f98964/volumes"
Sep 09 21:03:22.684143 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:22.684107    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:22.828175 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:22.828150    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:30.746809 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:30.746734    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:32.287592 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.287536    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:32.444295 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444243    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"]
Sep 09 21:03:32.444650 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444633    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main"
Sep 09 21:03:32.444743 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444653    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main"
Sep 09 21:03:32.444743 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444674    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy"
Sep 09 21:03:32.444743 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444683    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy"
Sep 09 21:03:32.444743 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444697    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy"
Sep 09 21:03:32.444743 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444706    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy"
Sep 09 21:03:32.444993 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444781    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy"
Sep 09 21:03:32.444993 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444794    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3dd029f-f42a-4796-a085-40fd1907f86e" containerName="istio-proxy"
Sep 09 21:03:32.444993 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.444805    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f27343a-c682-4725-86ef-4e07a1f98964" containerName="main"
Sep 09 21:03:32.449168 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.449147    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.453066 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.453042    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"openshift-service-ca.crt\""
Sep 09 21:03:32.453229 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.453099    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisvc-model-fb-opt-125m-oci-r-0fba19de-epp-sa-dockercfg-dd85l\""
Sep 09 21:03:32.453229 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.453097    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"kube-root-ca.crt\""
Sep 09 21:03:32.453398 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.453120    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"default-dockercfg-b9xtq\""
Sep 09 21:03:32.453398 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.453140    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisv20d1e8ef181cac7effb0fab7b8590b15-kserve-self-signed-certs\""
Sep 09 21:03:32.457709 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.457686    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"]
Sep 09 21:03:32.545233 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.545153    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:32.557617 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.557590    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl4mm\" (UniqueName: \"kubernetes.io/projected/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-kube-api-access-jl4mm\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.557728 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.557672    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.658494 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.658465    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.658606 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.658501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4mm\" (UniqueName: \"kubernetes.io/projected/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-kube-api-access-jl4mm\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.661081 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.661056    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-tls-certs\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.668358 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.668337    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4mm\" (UniqueName: \"kubernetes.io/projected/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-kube-api-access-jl4mm\") pod \"llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:32.759910 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:32.759888    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:33.033010 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:33.032987    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"]
Sep 09 21:03:33.034759 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:03:33.034734    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ce829d_ef09_48c1_bcbc_2b9c43b422e9.slice/crio-27a8528270d580280e62cda18095422a1f90e73e3d22f5478ad716b871098807 WatchSource:0}: Error finding container 27a8528270d580280e62cda18095422a1f90e73e3d22f5478ad716b871098807: Status 404 returned error can't find the container with id 27a8528270d580280e62cda18095422a1f90e73e3d22f5478ad716b871098807
Sep 09 21:03:33.534793 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:33.534759    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" event={"ID":"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9","Type":"ContainerStarted","Data":"2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4"}
Sep 09 21:03:33.535219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:33.534808    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" event={"ID":"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9","Type":"ContainerStarted","Data":"27a8528270d580280e62cda18095422a1f90e73e3d22f5478ad716b871098807"}
Sep 09 21:03:33.535219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:33.534891    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:03:33.560815 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:33.560778    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" podStartSLOduration=1.560764298 podStartE2EDuration="1.560764298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:33.558595479 +0000 UTC m=+3152.489678796" watchObservedRunningTime="2026-09-09 21:03:33.560764298 +0000 UTC m=+3152.491847615"
Sep 09 21:03:36.640345 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:36.640316    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:36.718847 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:36.718819    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:42.155415 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:03:42.155379    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:04.540539 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:04:04.540510    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:04:10.300595 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:04:10.300563    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:12.837445 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:04:12.837411    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:16.918386 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:04:16.918346    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:00.384604 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:00.384520    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:00.476535 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:00.476469    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:06.504450 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:06.504414    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:06.734270 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:06.734226    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:08.266651 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:08.266624    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:14.289389 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:14.289350    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:14.390333 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:14.390292    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:19.032462 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:19.032427    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:40.316316 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:40.316282    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:42.406003 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:42.405973    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:47.517154 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:47.517122    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:47.529799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:47.529767    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"]
Sep 09 21:05:47.618861 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:47.618818    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:54.540700 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:05:54.540657    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:06:01.728028 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:01.728000    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:06:01.735337 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:01.735316    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:06:01.740456 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:01.740433    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:06:01.746942 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:01.746924    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:06:02.552868 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:02.552809    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main" containerID="cri-o://2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4" gracePeriod=45
Sep 09 21:06:04.540525 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:04.540485    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:06:14.539827 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:14.539787    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:06:14.540364 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:14.539876    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:06:24.541076 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:24.540993    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:06:32.797058 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:32.797031    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:06:32.918948 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:32.918910    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl4mm\" (UniqueName: \"kubernetes.io/projected/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-kube-api-access-jl4mm\") pod \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") "
Sep 09 21:06:32.919102 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:32.918959    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-tls-certs\") pod \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\" (UID: \"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9\") "
Sep 09 21:06:32.921100 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:32.921069    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" (UID: "c6ce829d-ef09-48c1-bcbc-2b9c43b422e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:06:32.921202 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:32.921116    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-kube-api-access-jl4mm" (OuterVolumeSpecName: "kube-api-access-jl4mm") pod "c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" (UID: "c6ce829d-ef09-48c1-bcbc-2b9c43b422e9"). InnerVolumeSpecName "kube-api-access-jl4mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:06:33.019905 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.019879    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl4mm\" (UniqueName: \"kubernetes.io/projected/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-kube-api-access-jl4mm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:06:33.019905 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.019901    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:06:33.301839 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.301755    2573 generic.go:356] "Generic (PLEG): container finished" podID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerID="2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4" exitCode=0
Sep 09 21:06:33.301839 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.301805    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"
Sep 09 21:06:33.302043 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.301849    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" event={"ID":"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9","Type":"ContainerDied","Data":"2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4"}
Sep 09 21:06:33.302043 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.301917    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj" event={"ID":"c6ce829d-ef09-48c1-bcbc-2b9c43b422e9","Type":"ContainerDied","Data":"27a8528270d580280e62cda18095422a1f90e73e3d22f5478ad716b871098807"}
Sep 09 21:06:33.302043 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.301950    2573 scope.go:117] "RemoveContainer" containerID="2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4"
Sep 09 21:06:33.311553 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.311534    2573 scope.go:117] "RemoveContainer" containerID="2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4"
Sep 09 21:06:33.311805 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:06:33.311781    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4\": container with ID starting with 2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4 not found: ID does not exist" containerID="2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4"
Sep 09 21:06:33.311903 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.311808    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4"} err="failed to get container status \"2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4\": rpc error: code = NotFound desc = could not find container \"2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4\": container with ID starting with 2c539d6b4ded78b96b25b83d774733a0b85c5bb889113199fb6509b17e6b4da4 not found: ID does not exist"
Sep 09 21:06:33.324450 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.324420    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"]
Sep 09 21:06:33.331813 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.331790    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc20d1e8ef181cac7effb0fab7b8590b15-kserve-router-schetfqpj"]
Sep 09 21:06:33.641642 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:33.641612    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" path="/var/lib/kubelet/pods/c6ce829d-ef09-48c1-bcbc-2b9c43b422e9/volumes"
Sep 09 21:06:38.869328 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:38.869291    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:38.974310 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:38.974275    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:39.073658 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:39.073617    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:43.949210 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:43.949184    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:50.013550 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:50.013511    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:50.081548 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:50.081525    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:51.850453 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:51.850406    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:52.044351 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.044320    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"]
Sep 09 21:06:52.044646 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.044635    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main"
Sep 09 21:06:52.044691 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.044648    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main"
Sep 09 21:06:52.044725 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.044711    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ce829d-ef09-48c1-bcbc-2b9c43b422e9" containerName="main"
Sep 09 21:06:52.047299 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.047279    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.051128 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.051109    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Sep 09 21:06:52.051224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.051159    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Sep 09 21:06:52.051224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.051109    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-c2fdl\""
Sep 09 21:06:52.051224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.051112    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Sep 09 21:06:52.051224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.051117    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-jrpc7\""
Sep 09 21:06:52.057801 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.057780    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"]
Sep 09 21:06:52.076859 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.076841    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:52.168540 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.168519    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/720d9703-83d6-4464-8875-69b2f80461d9-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.168631 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.168566    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc2gk\" (UniqueName: \"kubernetes.io/projected/720d9703-83d6-4464-8875-69b2f80461d9-kube-api-access-tc2gk\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.269861 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.269835    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/720d9703-83d6-4464-8875-69b2f80461d9-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.269956 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.269884    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc2gk\" (UniqueName: \"kubernetes.io/projected/720d9703-83d6-4464-8875-69b2f80461d9-kube-api-access-tc2gk\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.272444 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.272424    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/720d9703-83d6-4464-8875-69b2f80461d9-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.278873 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.278850    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc2gk\" (UniqueName: \"kubernetes.io/projected/720d9703-83d6-4464-8875-69b2f80461d9-kube-api-access-tc2gk\") pod \"custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.358030 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.357997    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:52.627396 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:52.627371    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"]
Sep 09 21:06:52.629105 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:06:52.629067    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720d9703_83d6_4464_8875_69b2f80461d9.slice/crio-a21774daa2728e02ddfb1e2658eab89d423fec4cca6dbd0dea9e09507dbd88fe WatchSource:0}: Error finding container a21774daa2728e02ddfb1e2658eab89d423fec4cca6dbd0dea9e09507dbd88fe: Status 404 returned error can't find the container with id a21774daa2728e02ddfb1e2658eab89d423fec4cca6dbd0dea9e09507dbd88fe
Sep 09 21:06:53.397589 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:53.397556    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" event={"ID":"720d9703-83d6-4464-8875-69b2f80461d9","Type":"ContainerStarted","Data":"f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e"}
Sep 09 21:06:53.397954 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:53.397627    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" event={"ID":"720d9703-83d6-4464-8875-69b2f80461d9","Type":"ContainerStarted","Data":"a21774daa2728e02ddfb1e2658eab89d423fec4cca6dbd0dea9e09507dbd88fe"}
Sep 09 21:06:53.397954 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:53.397704    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:06:53.414977 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:53.414935    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" podStartSLOduration=1.414922758 podStartE2EDuration="1.414922758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:53.414202175 +0000 UTC m=+3352.345285495" watchObservedRunningTime="2026-09-09 21:06:53.414922758 +0000 UTC m=+3352.346006076"
Sep 09 21:06:54.632373 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:54.632327    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:54.711375 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:06:54.711343    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:07.163245 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:07.163209    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:24.404800 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:24.404768    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:07:43.138804 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:43.138753    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:43.246847 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:43.246812    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:45.383581 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:45.383543    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:45.483526 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:45.483491    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:45.585364 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:45.585315    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:49.941659 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:49.941624    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:51.828547 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:51.828505    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:51.927142 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:51.927113    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:56.035630 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:56.035556    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:56.133888 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:07:56.133857    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:00.189554 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:00.189517    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:00.472408 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:00.472316    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:03.303657 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:03.303615    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:03.422817 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:03.422781    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:03.990177 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:03.990144    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:04.091060 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:04.091028    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:04.190930 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:04.190896    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:08.410268 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:08.410226    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:08.510059 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:08.510023    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:08.610970 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:08.610936    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:09.057606 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:09.057572    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:13.030736 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:13.030697    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:13.123757 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:13.123721    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:15.335151 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:15.335121    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:15.480679 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:15.480644    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:19.511005 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:19.510969    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:19.576175 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:19.576144    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:29.683860 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:29.683823    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:29.919156 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:29.919122    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:32.609865 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:32.609835    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:32.704559 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:32.704526    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:33.885442 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:33.885414    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:33.980194 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:33.980163    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:35.811449 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:35.811418    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:38.957496 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:38.957458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:40.606783 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:40.606752    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:40.915112 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:40.915031    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:48.908183 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:48.908149    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:57.207475 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:57.207439    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:57.218420 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:57.218382    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"]
Sep 09 21:08:57.304506 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:08:57.304468    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:04.404083 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:04.404038    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:09:12.237760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:12.237711    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main" containerID="cri-o://f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e" gracePeriod=45
Sep 09 21:09:14.404058 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:14.404017    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:09:17.297865 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:17.297831    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:24.404027 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:24.403932    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:09:24.404482 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:24.404052    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:09:34.403815 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:34.403774    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:09:42.477117 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.477092    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:09:42.661507 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.661475    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc2gk\" (UniqueName: \"kubernetes.io/projected/720d9703-83d6-4464-8875-69b2f80461d9-kube-api-access-tc2gk\") pod \"720d9703-83d6-4464-8875-69b2f80461d9\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") "
Sep 09 21:09:42.661682 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.661551    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/720d9703-83d6-4464-8875-69b2f80461d9-tls-certs\") pod \"720d9703-83d6-4464-8875-69b2f80461d9\" (UID: \"720d9703-83d6-4464-8875-69b2f80461d9\") "
Sep 09 21:09:42.663647 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.663620    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720d9703-83d6-4464-8875-69b2f80461d9-kube-api-access-tc2gk" (OuterVolumeSpecName: "kube-api-access-tc2gk") pod "720d9703-83d6-4464-8875-69b2f80461d9" (UID: "720d9703-83d6-4464-8875-69b2f80461d9"). InnerVolumeSpecName "kube-api-access-tc2gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:42.663754 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.663654    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/720d9703-83d6-4464-8875-69b2f80461d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "720d9703-83d6-4464-8875-69b2f80461d9" (UID: "720d9703-83d6-4464-8875-69b2f80461d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:09:42.762383 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.762355    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/720d9703-83d6-4464-8875-69b2f80461d9-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:09:42.762383 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:42.762383    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc2gk\" (UniqueName: \"kubernetes.io/projected/720d9703-83d6-4464-8875-69b2f80461d9-kube-api-access-tc2gk\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:09:43.138171 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.138135    2573 generic.go:356] "Generic (PLEG): container finished" podID="720d9703-83d6-4464-8875-69b2f80461d9" containerID="f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e" exitCode=0
Sep 09 21:09:43.138370 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.138180    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"
Sep 09 21:09:43.138370 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.138230    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" event={"ID":"720d9703-83d6-4464-8875-69b2f80461d9","Type":"ContainerDied","Data":"f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e"}
Sep 09 21:09:43.138370 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.138313    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm" event={"ID":"720d9703-83d6-4464-8875-69b2f80461d9","Type":"ContainerDied","Data":"a21774daa2728e02ddfb1e2658eab89d423fec4cca6dbd0dea9e09507dbd88fe"}
Sep 09 21:09:43.138370 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.138341    2573 scope.go:117] "RemoveContainer" containerID="f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e"
Sep 09 21:09:43.147295 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.147279    2573 scope.go:117] "RemoveContainer" containerID="f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e"
Sep 09 21:09:43.147551 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:09:43.147531    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e\": container with ID starting with f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e not found: ID does not exist" containerID="f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e"
Sep 09 21:09:43.147618 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.147558    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e"} err="failed to get container status \"f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e\": rpc error: code = NotFound desc = could not find container \"f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e\": container with ID starting with f0639d73ef41babed411abcf2c7a735298cc7573e59013d891cd82a95a0e062e not found: ID does not exist"
Sep 09 21:09:43.160611 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.160587    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"]
Sep 09 21:09:43.164518 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.164495    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-54cbcc58m55sm"]
Sep 09 21:09:43.642135 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:43.642097    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="720d9703-83d6-4464-8875-69b2f80461d9" path="/var/lib/kubelet/pods/720d9703-83d6-4464-8875-69b2f80461d9/volumes"
Sep 09 21:09:47.291933 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.291894    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:47.428578 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.428545    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:47.521996 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.521964    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"]
Sep 09 21:09:47.522467 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.522446    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main"
Sep 09 21:09:47.522467 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.522466    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main"
Sep 09 21:09:47.522610 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.522572    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="720d9703-83d6-4464-8875-69b2f80461d9" containerName="main"
Sep 09 21:09:47.525934 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.525912    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.528755 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.528736    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Sep 09 21:09:47.528882 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.528859    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-tn682\""
Sep 09 21:09:47.529103 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.529080    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Sep 09 21:09:47.529598 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.529582    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Sep 09 21:09:47.542909 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.542847    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"]
Sep 09 21:09:47.569982 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.569958    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:47.597729 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597692    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-credential-socket\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.597841 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597761    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-certs\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.597841 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597805    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.597920 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597843    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-token\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.597920 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597873    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-podinfo\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.597920 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597898    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-socket\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.598015 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597954    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vxhq\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-kube-api-access-6vxhq\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.598015 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597980    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-envoy\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.598015 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.597996    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-data\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.663723 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.663694    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:47.698852 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698830    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-envoy\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.698927 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698856    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-data\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.698927 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-credential-socket\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.698927 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-certs\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699052 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699052 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698961    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-token\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699052 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.698996    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-podinfo\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699052 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-socket\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699240 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vxhq\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-kube-api-access-6vxhq\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699340 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699240    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-data\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699359    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-certs\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699544 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699522    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-socket\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699610 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699565    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-credential-socket\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.699711 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.699694    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.701086 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.701066    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-envoy\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.701401 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.701382    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-podinfo\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.709547 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.709529    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-token\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.709681 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.709664    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vxhq\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-kube-api-access-6vxhq\") pod \"router-gateway-1-openshift-default-785d98b7d8-92kcl\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:47.839437 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:47.839386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:48.114085 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:48.114060    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"]
Sep 09 21:09:48.116373 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:09:48.116344    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220b53fc_ab8f_4a2a_aea9_579b5eb0f728.slice/crio-8cac2a613732052fbd43d029bf67e9570928a6ce41691e14feb46fffc5fb56ff WatchSource:0}: Error finding container 8cac2a613732052fbd43d029bf67e9570928a6ce41691e14feb46fffc5fb56ff: Status 404 returned error can't find the container with id 8cac2a613732052fbd43d029bf67e9570928a6ce41691e14feb46fffc5fb56ff
Sep 09 21:09:48.118501 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:48.118471    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:09:48.118574 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:48.118537    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:09:48.118574 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:48.118564    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:09:48.160721 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:48.160700    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" event={"ID":"220b53fc-ab8f-4a2a-aea9-579b5eb0f728","Type":"ContainerStarted","Data":"8cac2a613732052fbd43d029bf67e9570928a6ce41691e14feb46fffc5fb56ff"}
Sep 09 21:09:49.167116 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:49.167085    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" event={"ID":"220b53fc-ab8f-4a2a-aea9-579b5eb0f728","Type":"ContainerStarted","Data":"7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"}
Sep 09 21:09:49.189357 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:49.189310    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podStartSLOduration=2.189295991 podStartE2EDuration="2.189295991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:49.187920192 +0000 UTC m=+3528.119003512" watchObservedRunningTime="2026-09-09 21:09:49.189295991 +0000 UTC m=+3528.120379299"
Sep 09 21:09:49.839667 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:49.839639    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:49.844046 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:49.844023    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:50.171563 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:50.171534    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:50.172427 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:50.172411    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:09:51.306577 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:51.306541    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:54.544106 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:54.544062    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:59.217478 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:59.217378    2573 generic.go:356] "Generic (PLEG): container finished" podID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerID="7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab" exitCode=0
Sep 09 21:09:59.217478 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:59.217419    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" event={"ID":"220b53fc-ab8f-4a2a-aea9-579b5eb0f728","Type":"ContainerDied","Data":"7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"}
Sep 09 21:09:59.217983 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:59.217737    2573 scope.go:117] "RemoveContainer" containerID="7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"
Sep 09 21:09:59.218638 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:59.218610    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:09:59.218717 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:59.218681    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:09:59.218777 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:09:59.218730    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:10:00.224113 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:00.224081    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" event={"ID":"220b53fc-ab8f-4a2a-aea9-579b5eb0f728","Type":"ContainerStarted","Data":"b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b"}
Sep 09 21:10:00.840480 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:00.840447    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:10:00.841343 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:00.841313    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 21:10:00.881305 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:00.881280    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:00.979392 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:00.979362    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:01.840226 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:01.840187    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 21:10:02.840589 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:02.840549    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 21:10:03.840610 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:03.840563    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 21:10:04.840387 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:04.840350    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 21:10:05.172267 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:05.172212    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:10:05.843918 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:05.843890    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:10:05.845027 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:05.845008    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:10:07.354581 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:07.354541    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:07.953779 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:07.953737    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:08.204598 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:08.204506    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:49.082286 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:49.082246    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:53.165002 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:53.164971    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:53.265529 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:53.265493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:53.364968 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:53.364937    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:59.697578 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.697494    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:59.985328 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.985234    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"]
Sep 09 21:10:59.993374 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.993332    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:10:59.996943 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.996918    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"]
Sep 09 21:10:59.997659 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.997635    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Sep 09 21:10:59.997659 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.997649    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Sep 09 21:10:59.997848 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.997684    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-ww8kj\""
Sep 09 21:10:59.997848 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.997709    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Sep 09 21:10:59.998012 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:10:59.997996    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-77kvx\""
Sep 09 21:11:00.014690 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.014662    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:00.075729 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.075698    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9633d21c-03f4-4725-807f-f443525b9912-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.075848 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.075771    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n577d\" (UniqueName: \"kubernetes.io/projected/9633d21c-03f4-4725-807f-f443525b9912-kube-api-access-n577d\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.176755 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.176730    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9633d21c-03f4-4725-807f-f443525b9912-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.176869 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.176770    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n577d\" (UniqueName: \"kubernetes.io/projected/9633d21c-03f4-4725-807f-f443525b9912-kube-api-access-n577d\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.179146 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.179128    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9633d21c-03f4-4725-807f-f443525b9912-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.186623 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.186603    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n577d\" (UniqueName: \"kubernetes.io/projected/9633d21c-03f4-4725-807f-f443525b9912-kube-api-access-n577d\") pod \"stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.304963 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.304905    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:00.575780 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:00.575757    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"]
Sep 09 21:11:00.577996 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:11:00.577966    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9633d21c_03f4_4725_807f_f443525b9912.slice/crio-44ac5793103776af2d51de16f1a0229c797fe8c2fdbcd520672ca684f3508c9a WatchSource:0}: Error finding container 44ac5793103776af2d51de16f1a0229c797fe8c2fdbcd520672ca684f3508c9a: Status 404 returned error can't find the container with id 44ac5793103776af2d51de16f1a0229c797fe8c2fdbcd520672ca684f3508c9a
Sep 09 21:11:01.496585 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.496555    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" event={"ID":"9633d21c-03f4-4725-807f-f443525b9912","Type":"ContainerStarted","Data":"37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079"}
Sep 09 21:11:01.496980 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.496625    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" event={"ID":"9633d21c-03f4-4725-807f-f443525b9912","Type":"ContainerStarted","Data":"44ac5793103776af2d51de16f1a0229c797fe8c2fdbcd520672ca684f3508c9a"}
Sep 09 21:11:01.496980 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.496682    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:11:01.514148 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.514100    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podStartSLOduration=2.514086324 podStartE2EDuration="2.514086324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:01.512926736 +0000 UTC m=+3600.444010058" watchObservedRunningTime="2026-09-09 21:11:01.514086324 +0000 UTC m=+3600.445169643"
Sep 09 21:11:01.755749 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.755675    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:11:01.762661 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.762640    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:11:01.767774 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.767757    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:11:01.774539 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:01.774522    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:11:18.354234 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:18.354194    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:32.502453 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:11:32.502420    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:12:12.466409 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:12:12.466374    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:12.529391 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:12:12.529352    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:01.881287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:01.881190    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"]
Sep 09 21:13:01.892549 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:01.892518    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:01.948680 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:01.948652    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:02.267748 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:02.267675    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:02.366770 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:02.366741    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:02.502210 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:02.502172    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:13:07.223132 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:07.223095    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"]
Sep 09 21:13:07.223648 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:07.223322    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy" containerID="cri-o://b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b" gracePeriod=30
Sep 09 21:13:07.270570 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:07.270545    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:12.469910 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.469886    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:13:12.502301 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.502224    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:13:12.517064 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517040    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-socket\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517206 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517073    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-credential-socket\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517206 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517130    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-certs\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517206 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517151    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istiod-ca-cert\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517206 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517197    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-token\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517436 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517222    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-envoy\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517436 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517344    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:12.517550 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517430    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:12.517550 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517443    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:12.517550 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517488    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vxhq\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-kube-api-access-6vxhq\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517550 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517528    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-data\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517769 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517556    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-podinfo\") pod \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\" (UID: \"220b53fc-ab8f-4a2a-aea9-579b5eb0f728\") "
Sep 09 21:13:12.517769 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517586    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:13:12.517874 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.517861    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-data" (OuterVolumeSpecName: "istio-data") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:12.518132 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.518110    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.518224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.518136    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-credential-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.518224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.518149    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-workload-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.518224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.518164    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istiod-ca-cert\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.518224 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.518175    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-data\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.520115 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.520084    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-kube-api-access-6vxhq" (OuterVolumeSpecName: "kube-api-access-6vxhq") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "kube-api-access-6vxhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:13:12.520432 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.520408    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:13:12.520432 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.520410    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:12.520579 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.520561    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-token" (OuterVolumeSpecName: "istio-token") pod "220b53fc-ab8f-4a2a-aea9-579b5eb0f728" (UID: "220b53fc-ab8f-4a2a-aea9-579b5eb0f728"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:13:12.618851 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.618821    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-token\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.618851 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.618847    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-envoy\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.618989 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.618859    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vxhq\" (UniqueName: \"kubernetes.io/projected/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-kube-api-access-6vxhq\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:12.618989 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:12.618872    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220b53fc-ab8f-4a2a-aea9-579b5eb0f728-istio-podinfo\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:13.084458 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.084426    2573 generic.go:356] "Generic (PLEG): container finished" podID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerID="b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b" exitCode=0
Sep 09 21:13:13.084640 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.084478    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"
Sep 09 21:13:13.084640 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.084496    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" event={"ID":"220b53fc-ab8f-4a2a-aea9-579b5eb0f728","Type":"ContainerDied","Data":"b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b"}
Sep 09 21:13:13.084640 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.084539    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl" event={"ID":"220b53fc-ab8f-4a2a-aea9-579b5eb0f728","Type":"ContainerDied","Data":"8cac2a613732052fbd43d029bf67e9570928a6ce41691e14feb46fffc5fb56ff"}
Sep 09 21:13:13.084640 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.084561    2573 scope.go:117] "RemoveContainer" containerID="b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b"
Sep 09 21:13:13.093225 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.093209    2573 scope.go:117] "RemoveContainer" containerID="7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"
Sep 09 21:13:13.100226 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.100210    2573 scope.go:117] "RemoveContainer" containerID="b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b"
Sep 09 21:13:13.100490 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:13:13.100474    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b\": container with ID starting with b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b not found: ID does not exist" containerID="b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b"
Sep 09 21:13:13.100544 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.100495    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b"} err="failed to get container status \"b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b\": rpc error: code = NotFound desc = could not find container \"b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b\": container with ID starting with b5550de2c1f3046263a4104786768ed7590f32e6a9fbacc86fe2086d1b4d332b not found: ID does not exist"
Sep 09 21:13:13.100544 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.100510    2573 scope.go:117] "RemoveContainer" containerID="7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"
Sep 09 21:13:13.100738 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:13:13.100719    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab\": container with ID starting with 7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab not found: ID does not exist" containerID="7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"
Sep 09 21:13:13.100784 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.100743    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab"} err="failed to get container status \"7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab\": rpc error: code = NotFound desc = could not find container \"7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab\": container with ID starting with 7dcdcd0ed39079ea886a74194b915ee610d75283ab6411403ff0e0f73c691bab not found: ID does not exist"
Sep 09 21:13:13.106749 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.106730    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"]
Sep 09 21:13:13.113353 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.113333    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-785d98b7d8-92kcl"]
Sep 09 21:13:13.641200 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:13.641171    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" path="/var/lib/kubelet/pods/220b53fc-ab8f-4a2a-aea9-579b5eb0f728/volumes"
Sep 09 21:13:14.009091 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:14.008972    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:14.292458 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:14.292359    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:16.902015 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:16.901974    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main" containerID="cri-o://37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079" gracePeriod=45
Sep 09 21:13:22.503434 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:22.503395    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:13:22.503886 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:22.503490    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:13:29.036769 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:29.036737    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:30.436017 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:30.435979    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:30.897389 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:30.897350    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:32.501932 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:32.501893    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:13:36.688748 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:36.688719    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:42.502904 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:42.502868    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:13:47.143557 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.143528    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:13:47.240891 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.240808    2573 generic.go:356] "Generic (PLEG): container finished" podID="9633d21c-03f4-4725-807f-f443525b9912" containerID="37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079" exitCode=0
Sep 09 21:13:47.240891 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.240854    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"
Sep 09 21:13:47.241057 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.240891    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" event={"ID":"9633d21c-03f4-4725-807f-f443525b9912","Type":"ContainerDied","Data":"37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079"}
Sep 09 21:13:47.241057 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.240966    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h" event={"ID":"9633d21c-03f4-4725-807f-f443525b9912","Type":"ContainerDied","Data":"44ac5793103776af2d51de16f1a0229c797fe8c2fdbcd520672ca684f3508c9a"}
Sep 09 21:13:47.241057 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.240991    2573 scope.go:117] "RemoveContainer" containerID="37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079"
Sep 09 21:13:47.249311 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.249293    2573 scope.go:117] "RemoveContainer" containerID="37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079"
Sep 09 21:13:47.249579 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:13:47.249562    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079\": container with ID starting with 37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079 not found: ID does not exist" containerID="37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079"
Sep 09 21:13:47.249634 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.249588    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079"} err="failed to get container status \"37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079\": rpc error: code = NotFound desc = could not find container \"37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079\": container with ID starting with 37551c24a293b339ec333c80b795c2e2111a66902f1e81f44488970d5560f079 not found: ID does not exist"
Sep 09 21:13:47.289846 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.289821    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9633d21c-03f4-4725-807f-f443525b9912-tls-certs\") pod \"9633d21c-03f4-4725-807f-f443525b9912\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") "
Sep 09 21:13:47.289956 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.289851    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n577d\" (UniqueName: \"kubernetes.io/projected/9633d21c-03f4-4725-807f-f443525b9912-kube-api-access-n577d\") pod \"9633d21c-03f4-4725-807f-f443525b9912\" (UID: \"9633d21c-03f4-4725-807f-f443525b9912\") "
Sep 09 21:13:47.291904 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.291877    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9633d21c-03f4-4725-807f-f443525b9912-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9633d21c-03f4-4725-807f-f443525b9912" (UID: "9633d21c-03f4-4725-807f-f443525b9912"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:13:47.292051 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.292030    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9633d21c-03f4-4725-807f-f443525b9912-kube-api-access-n577d" (OuterVolumeSpecName: "kube-api-access-n577d") pod "9633d21c-03f4-4725-807f-f443525b9912" (UID: "9633d21c-03f4-4725-807f-f443525b9912"). InnerVolumeSpecName "kube-api-access-n577d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:13:47.391002 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.390978    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9633d21c-03f4-4725-807f-f443525b9912-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:47.391002 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.391001    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n577d\" (UniqueName: \"kubernetes.io/projected/9633d21c-03f4-4725-807f-f443525b9912-kube-api-access-n577d\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:13:47.562499 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.562463    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"]
Sep 09 21:13:47.571587 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.571559    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-7cc9bbb8f8-7624h"]
Sep 09 21:13:47.642117 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:47.642085    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9633d21c-03f4-4725-807f-f443525b9912" path="/var/lib/kubelet/pods/9633d21c-03f4-4725-807f-f443525b9912/volumes"
Sep 09 21:13:51.420460 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:51.420426    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:51.520571 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:51.520537    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:51.628861 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:51.628830    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:55.741079 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:55.741054    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:57.968742 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:13:57.968703    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:01.780382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:14:01.780346    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:01.880994 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:14:01.880953    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:04.217327 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:14:04.217292    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:04.376589 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:14:04.376555    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:08.285119 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:14:08.285081    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:17.508537 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:14:17.508496    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:34.444395 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:15:34.444317    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:36.606479 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:15:36.606445    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:59.561471 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:15:59.561434    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:59.622614 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:15:59.622586    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:01.782367 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:01.782341    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:16:01.791332 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:01.791310    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:16:01.795004 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:01.794984    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:16:01.803152 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:01.803134    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:16:13.065081 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:13.065032    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:24.585100 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:24.585066    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:24.684430 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:24.684395    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.205613 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:31.205573    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:33.475526 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:33.475489    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:33.576966 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:33.576928    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:33.676491 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:33.676459    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:35.575792 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.575758    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:35.842223 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842141    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"]
Sep 09 21:16:35.842516 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842502    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy"
Sep 09 21:16:35.842568 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842518    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy"
Sep 09 21:16:35.842568 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842531    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy"
Sep 09 21:16:35.842568 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842536    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy"
Sep 09 21:16:35.842568 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842550    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main"
Sep 09 21:16:35.842568 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842555    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main"
Sep 09 21:16:35.842893 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842611    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy"
Sep 09 21:16:35.842893 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842621    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="220b53fc-ab8f-4a2a-aea9-579b5eb0f728" containerName="istio-proxy"
Sep 09 21:16:35.842893 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.842629    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9633d21c-03f4-4725-807f-f443525b9912" containerName="main"
Sep 09 21:16:35.845484 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.845463    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:35.848078 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.848058    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Sep 09 21:16:35.848952 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.848936    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-gggfh\""
Sep 09 21:16:35.849088 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.849069    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Sep 09 21:16:35.849168 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.849132    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-pkls4\""
Sep 09 21:16:35.849288 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.849272    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Sep 09 21:16:35.856596 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.856575    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"]
Sep 09 21:16:35.872062 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:35.872038    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:36.006915 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.006866    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f026575f-3566-4fa4-9b09-8a9a09d8ccac-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.007089 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.006971    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsgzw\" (UniqueName: \"kubernetes.io/projected/f026575f-3566-4fa4-9b09-8a9a09d8ccac-kube-api-access-wsgzw\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.107933 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.107879    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsgzw\" (UniqueName: \"kubernetes.io/projected/f026575f-3566-4fa4-9b09-8a9a09d8ccac-kube-api-access-wsgzw\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.108118 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.107974    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f026575f-3566-4fa4-9b09-8a9a09d8ccac-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.110596 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.110570    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f026575f-3566-4fa4-9b09-8a9a09d8ccac-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.117510 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.117488    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsgzw\" (UniqueName: \"kubernetes.io/projected/f026575f-3566-4fa4-9b09-8a9a09d8ccac-kube-api-access-wsgzw\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.155386 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.155356    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.429410 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.429323    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"]
Sep 09 21:16:36.432470 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:16:36.432439    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf026575f_3566_4fa4_9b09_8a9a09d8ccac.slice/crio-6a6aae0edd0f6a069fc545c6adf5216ab30ea9cd50efb0be287ded03ed38a534 WatchSource:0}: Error finding container 6a6aae0edd0f6a069fc545c6adf5216ab30ea9cd50efb0be287ded03ed38a534: Status 404 returned error can't find the container with id 6a6aae0edd0f6a069fc545c6adf5216ab30ea9cd50efb0be287ded03ed38a534
Sep 09 21:16:36.975736 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.975705    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" event={"ID":"f026575f-3566-4fa4-9b09-8a9a09d8ccac","Type":"ContainerStarted","Data":"f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34"}
Sep 09 21:16:36.976176 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.975756    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" event={"ID":"f026575f-3566-4fa4-9b09-8a9a09d8ccac","Type":"ContainerStarted","Data":"6a6aae0edd0f6a069fc545c6adf5216ab30ea9cd50efb0be287ded03ed38a534"}
Sep 09 21:16:36.976176 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.975795    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:16:36.993854 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:36.993663    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podStartSLOduration=1.99364479 podStartE2EDuration="1.99364479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:36.993341943 +0000 UTC m=+3935.924425261" watchObservedRunningTime="2026-09-09 21:16:36.99364479 +0000 UTC m=+3935.924728111"
Sep 09 21:16:37.785970 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:37.785935    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:37.892759 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:37.892725    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:40.339461 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:40.339424    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:40.400684 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:40.400652    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:44.158307 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:44.158270    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:50.683945 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:50.683908    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:56.425999 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:16:56.425914    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:17:07.981573 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:17:07.981540    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:17:10.652306 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:17:10.652263    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:20.209835 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:18:20.209801    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:09.385592 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:09.385552    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:15.456598 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:15.456563    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:15.476597 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:15.476562    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"]
Sep 09 21:19:15.524912 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:15.524882    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:17.981595 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:17.981559    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:19:27.982006 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:27.981972    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:19:29.497503 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:29.497468    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:29.599962 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:29.599917    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:30.496509 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:30.496469    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main" containerID="cri-o://f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34" gracePeriod=45
Sep 09 21:19:35.203593 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:35.203550    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:35.408537 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:35.408493    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:37.040180 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:37.040146    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:37.139243 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:37.139205    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:37.981313 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:37.981280    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:19:37.981476 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:37.981384    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:19:40.179073 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:40.179039    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:41.177712 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:41.177680    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:47.981192 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:47.981152    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:19:57.981701 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:19:57.981622    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:20:00.738061 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.738038    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:20:00.855760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.855731    2573 generic.go:356] "Generic (PLEG): container finished" podID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerID="f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34" exitCode=0
Sep 09 21:20:00.855891 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.855772    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"
Sep 09 21:20:00.855891 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.855817    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" event={"ID":"f026575f-3566-4fa4-9b09-8a9a09d8ccac","Type":"ContainerDied","Data":"f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34"}
Sep 09 21:20:00.855891 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.855873    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv" event={"ID":"f026575f-3566-4fa4-9b09-8a9a09d8ccac","Type":"ContainerDied","Data":"6a6aae0edd0f6a069fc545c6adf5216ab30ea9cd50efb0be287ded03ed38a534"}
Sep 09 21:20:00.856013 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.855894    2573 scope.go:117] "RemoveContainer" containerID="f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34"
Sep 09 21:20:00.863602 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.863585    2573 scope.go:117] "RemoveContainer" containerID="f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34"
Sep 09 21:20:00.863851 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:20:00.863833    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34\": container with ID starting with f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34 not found: ID does not exist" containerID="f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34"
Sep 09 21:20:00.863898 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.863860    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34"} err="failed to get container status \"f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34\": rpc error: code = NotFound desc = could not find container \"f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34\": container with ID starting with f68bd8fbb531ff2d3f1a0281f083ded8253846bbdfb49de781efcbca0ea70d34 not found: ID does not exist"
Sep 09 21:20:00.888208 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.888189    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f026575f-3566-4fa4-9b09-8a9a09d8ccac-tls-certs\") pod \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") "
Sep 09 21:20:00.888316 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.888304    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsgzw\" (UniqueName: \"kubernetes.io/projected/f026575f-3566-4fa4-9b09-8a9a09d8ccac-kube-api-access-wsgzw\") pod \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\" (UID: \"f026575f-3566-4fa4-9b09-8a9a09d8ccac\") "
Sep 09 21:20:00.890370 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.890348    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f026575f-3566-4fa4-9b09-8a9a09d8ccac-kube-api-access-wsgzw" (OuterVolumeSpecName: "kube-api-access-wsgzw") pod "f026575f-3566-4fa4-9b09-8a9a09d8ccac" (UID: "f026575f-3566-4fa4-9b09-8a9a09d8ccac"). InnerVolumeSpecName "kube-api-access-wsgzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:20:00.890459 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.890391    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f026575f-3566-4fa4-9b09-8a9a09d8ccac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f026575f-3566-4fa4-9b09-8a9a09d8ccac" (UID: "f026575f-3566-4fa4-9b09-8a9a09d8ccac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:20:00.988993 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.988958    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsgzw\" (UniqueName: \"kubernetes.io/projected/f026575f-3566-4fa4-9b09-8a9a09d8ccac-kube-api-access-wsgzw\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:20:00.988993 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:00.988991    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f026575f-3566-4fa4-9b09-8a9a09d8ccac-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:20:01.181850 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:01.181822    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"]
Sep 09 21:20:01.188740 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:01.188709    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-84c48wbdbv"]
Sep 09 21:20:01.643533 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:01.643498    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" path="/var/lib/kubelet/pods/f026575f-3566-4fa4-9b09-8a9a09d8ccac/volumes"
Sep 09 21:20:05.680847 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:05.680816    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:05.818750 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:05.818718    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:06.062049 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.061974    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"]
Sep 09 21:20:06.062345 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.062325    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main"
Sep 09 21:20:06.062345 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.062341    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main"
Sep 09 21:20:06.062490 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.062411    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f026575f-3566-4fa4-9b09-8a9a09d8ccac" containerName="main"
Sep 09 21:20:06.066771 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.066751    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.072168 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.072142    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-gw-ca-root-cert\""
Sep 09 21:20:06.072311 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.072281    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Sep 09 21:20:06.073218 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.073196    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Sep 09 21:20:06.073629 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.073326    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-gateway-2-openshift-default-dockercfg-dtr42\""
Sep 09 21:20:06.089654 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.089627    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"]
Sep 09 21:20:06.231780 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.231753    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-envoy\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.231911 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.231785    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-credential-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.231911 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.231812    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-token\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.231911 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.231893    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-data\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.232014 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.231920    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks4m5\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-kube-api-access-ks4m5\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.232014 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.231982    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3045579-a8ca-4093-ab53-e27ca10bba9a-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.232014 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.232006    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.232108 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.232025    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-certs\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.232108 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.232079    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-podinfo\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333045 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.332983    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-token\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333045 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333017    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-data\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333045 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333038    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks4m5\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-kube-api-access-ks4m5\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3045579-a8ca-4093-ab53-e27ca10bba9a-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333091    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333107    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-certs\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333127    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-podinfo\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333148    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-envoy\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333382 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-credential-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333673 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333456    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-data\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333673 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333539    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333783 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333674    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-credential-socket\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333783 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333696    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-certs\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.333906 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.333887    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3045579-a8ca-4093-ab53-e27ca10bba9a-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.335507 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.335480    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-envoy\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.335698 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.335680    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-podinfo\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.341778 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.341756    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-token\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.341932 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.341912    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks4m5\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-kube-api-access-ks4m5\") pod \"router-gateway-2-openshift-default-5858dff66d-k6fqg\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.377110 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.377090    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:06.471061 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.471031    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:06.596284 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.596191    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:06.654297 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.654269    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"]
Sep 09 21:20:06.657001 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:20:06.656975    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3045579_a8ca_4093_ab53_e27ca10bba9a.slice/crio-4132475a9a944e02b9b9a19187d7d67dce029a7dab92f6a8676574455a900169 WatchSource:0}: Error finding container 4132475a9a944e02b9b9a19187d7d67dce029a7dab92f6a8676574455a900169: Status 404 returned error can't find the container with id 4132475a9a944e02b9b9a19187d7d67dce029a7dab92f6a8676574455a900169
Sep 09 21:20:06.658956 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.658922    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:20:06.659044 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.659025    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:20:06.659084 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.659063    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:20:06.885277 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.885229    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" event={"ID":"f3045579-a8ca-4093-ab53-e27ca10bba9a","Type":"ContainerStarted","Data":"5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"}
Sep 09 21:20:06.885725 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.885298    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" event={"ID":"f3045579-a8ca-4093-ab53-e27ca10bba9a","Type":"ContainerStarted","Data":"4132475a9a944e02b9b9a19187d7d67dce029a7dab92f6a8676574455a900169"}
Sep 09 21:20:06.906079 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:06.906032    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podStartSLOduration=0.906017326 podStartE2EDuration="906.017326ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:20:06.904990372 +0000 UTC m=+4145.836073691" watchObservedRunningTime="2026-09-09 21:20:06.906017326 +0000 UTC m=+4145.837100648"
Sep 09 21:20:07.377917 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:07.377888    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:07.379108 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:07.379080    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 09 21:20:08.381123 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:08.381090    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:08.896465 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:08.896426    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:08.953553 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:08.953478    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:09.484335 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:09.484304    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:12.928866 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:12.928832    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:17.934699 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:17.934663    2573 generic.go:356] "Generic (PLEG): container finished" podID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerID="5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7" exitCode=0
Sep 09 21:20:17.935130 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:17.934734    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" event={"ID":"f3045579-a8ca-4093-ab53-e27ca10bba9a","Type":"ContainerDied","Data":"5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"}
Sep 09 21:20:17.935130 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:17.935081    2573 scope.go:117] "RemoveContainer" containerID="5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"
Sep 09 21:20:17.935951 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:17.935927    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:20:17.936056 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:17.935992    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:20:17.936056 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:17.936026    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 09 21:20:18.942001 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:18.941961    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" event={"ID":"f3045579-a8ca-4093-ab53-e27ca10bba9a","Type":"ContainerStarted","Data":"228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae"}
Sep 09 21:20:19.338080 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:19.338000    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:19.377840 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:19.377810    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:19.379027 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:19.378998    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 09 21:20:19.439072 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:19.439046    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:20.091109 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.091073    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:20.338758 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.338722    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"]
Sep 09 21:20:20.341062 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.341039    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.343713 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.343664    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-886t2\""
Sep 09 21:20:20.344658 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.344642    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-bwn6w\""
Sep 09 21:20:20.344760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.344732    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Sep 09 21:20:20.359486 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.358010    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"]
Sep 09 21:20:20.377726 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.377686    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 09 21:20:20.378666 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.378641    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d3769e-23e8-40b3-939c-5d1bafda539d-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.378890 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.378871    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xr4\" (UniqueName: \"kubernetes.io/projected/84d3769e-23e8-40b3-939c-5d1bafda539d-kube-api-access-d2xr4\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.388545 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.388518    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:20.480448 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.480409    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xr4\" (UniqueName: \"kubernetes.io/projected/84d3769e-23e8-40b3-939c-5d1bafda539d-kube-api-access-d2xr4\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.480604 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.480485    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d3769e-23e8-40b3-939c-5d1bafda539d-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.482985 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.482964    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d3769e-23e8-40b3-939c-5d1bafda539d-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.485841 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.485816    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:20.491202 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.491180    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xr4\" (UniqueName: \"kubernetes.io/projected/84d3769e-23e8-40b3-939c-5d1bafda539d-kube-api-access-d2xr4\") pod \"router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.652377 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.652345    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:20.939279 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.936804    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"]
Sep 09 21:20:20.942315 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:20:20.942282    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d3769e_23e8_40b3_939c_5d1bafda539d.slice/crio-1c27f04025da3fcd4f0bdf80d1362434c3654efee4a3f8ec460ec9794d2001c5 WatchSource:0}: Error finding container 1c27f04025da3fcd4f0bdf80d1362434c3654efee4a3f8ec460ec9794d2001c5: Status 404 returned error can't find the container with id 1c27f04025da3fcd4f0bdf80d1362434c3654efee4a3f8ec460ec9794d2001c5
Sep 09 21:20:20.975061 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:20.975024    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" event={"ID":"84d3769e-23e8-40b3-939c-5d1bafda539d","Type":"ContainerStarted","Data":"1c27f04025da3fcd4f0bdf80d1362434c3654efee4a3f8ec460ec9794d2001c5"}
Sep 09 21:20:21.377667 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:21.377631    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 09 21:20:21.980718 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:21.980686    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" event={"ID":"84d3769e-23e8-40b3-939c-5d1bafda539d","Type":"ContainerStarted","Data":"1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec"}
Sep 09 21:20:21.980924 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:21.980834    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:20:22.378136 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:22.378091    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 09 21:20:23.378106 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:23.378063    2573 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 09 21:20:23.896579 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:23.896552    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:24.381508 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:24.381481    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:24.382515 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:24.382492    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:20:24.412615 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:24.412566    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podStartSLOduration=4.412551671 podStartE2EDuration="4.412551671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:20:22.000394612 +0000 UTC m=+4160.931477954" watchObservedRunningTime="2026-09-09 21:20:24.412551671 +0000 UTC m=+4163.343634988"
Sep 09 21:20:49.886830 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:49.886796    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:51.256047 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:51.256012    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:52.986006 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:20:52.985978    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:21:01.809809 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:21:01.809784    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:21:01.822952 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:21:01.822929    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:21:01.825670 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:21:01.825647    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:21:01.834808 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:21:01.834784    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:21:43.769109 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:21:43.769031    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:43.869009 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:21:43.868976    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:29.689109 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:29.689074    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:29.708322 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:29.708289    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"]
Sep 09 21:22:29.773847 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:29.773814    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:32.986620 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:32.986581    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:22:33.926124 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:33.926090    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:34.021521 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:34.021488    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:38.038843 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:38.038808    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:38.296831 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:38.296747    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:41.652912 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:41.652881    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:42.986350 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:42.986308    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:22:44.727572 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:44.727528    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main" containerID="cri-o://1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec" gracePeriod=45
Sep 09 21:22:47.380355 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:47.380322    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:48.720467 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:48.720433    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:48.814073 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:48.814044    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:52.986060 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:52.986017    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:22:52.986490 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:22:52.986125    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:23:02.986555 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:02.986515    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:23:12.986152 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:12.986110    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Sep 09 21:23:14.971052 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:14.971029    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:23:15.090061 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.089996    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d3769e-23e8-40b3-939c-5d1bafda539d-tls-certs\") pod \"84d3769e-23e8-40b3-939c-5d1bafda539d\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") "
Sep 09 21:23:15.090061 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.090056    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2xr4\" (UniqueName: \"kubernetes.io/projected/84d3769e-23e8-40b3-939c-5d1bafda539d-kube-api-access-d2xr4\") pod \"84d3769e-23e8-40b3-939c-5d1bafda539d\" (UID: \"84d3769e-23e8-40b3-939c-5d1bafda539d\") "
Sep 09 21:23:15.092127 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.092099    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d3769e-23e8-40b3-939c-5d1bafda539d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84d3769e-23e8-40b3-939c-5d1bafda539d" (UID: "84d3769e-23e8-40b3-939c-5d1bafda539d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:23:15.092241 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.092145    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d3769e-23e8-40b3-939c-5d1bafda539d-kube-api-access-d2xr4" (OuterVolumeSpecName: "kube-api-access-d2xr4") pod "84d3769e-23e8-40b3-939c-5d1bafda539d" (UID: "84d3769e-23e8-40b3-939c-5d1bafda539d"). InnerVolumeSpecName "kube-api-access-d2xr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:23:15.191496 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.191468    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2xr4\" (UniqueName: \"kubernetes.io/projected/84d3769e-23e8-40b3-939c-5d1bafda539d-kube-api-access-d2xr4\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:15.191496 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.191492    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84d3769e-23e8-40b3-939c-5d1bafda539d-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:15.746288 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.746232    2573 generic.go:356] "Generic (PLEG): container finished" podID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerID="1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec" exitCode=0
Sep 09 21:23:15.746458 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.746301    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"
Sep 09 21:23:15.746458 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.746319    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" event={"ID":"84d3769e-23e8-40b3-939c-5d1bafda539d","Type":"ContainerDied","Data":"1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec"}
Sep 09 21:23:15.746458 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.746372    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr" event={"ID":"84d3769e-23e8-40b3-939c-5d1bafda539d","Type":"ContainerDied","Data":"1c27f04025da3fcd4f0bdf80d1362434c3654efee4a3f8ec460ec9794d2001c5"}
Sep 09 21:23:15.746458 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.746397    2573 scope.go:117] "RemoveContainer" containerID="1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec"
Sep 09 21:23:15.754271 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.754235    2573 scope.go:117] "RemoveContainer" containerID="1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec"
Sep 09 21:23:15.754574 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:23:15.754556    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec\": container with ID starting with 1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec not found: ID does not exist" containerID="1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec"
Sep 09 21:23:15.754648 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.754580    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec"} err="failed to get container status \"1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec\": rpc error: code = NotFound desc = could not find container \"1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec\": container with ID starting with 1bf51e56ac83aef538a248cbf6275df9c09ded2a82e7f085720427570ccbccec not found: ID does not exist"
Sep 09 21:23:15.765708 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.765685    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"]
Sep 09 21:23:15.769877 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:15.769860    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-99c668b45mlnhr"]
Sep 09 21:23:17.642096 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:17.642065    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" path="/var/lib/kubelet/pods/84d3769e-23e8-40b3-939c-5d1bafda539d/volumes"
Sep 09 21:23:19.774905 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:19.774873    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:19.876204 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:19.876174    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:24.735967 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:24.735928    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"]
Sep 09 21:23:24.736470 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:24.736167    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy" containerID="cri-o://228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae" gracePeriod=30
Sep 09 21:23:24.795049 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:24.795022    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:29.580529 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.580499    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"]
Sep 09 21:23:29.580901 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.580862    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main"
Sep 09 21:23:29.580901 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.580874    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main"
Sep 09 21:23:29.580979 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.580933    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d3769e-23e8-40b3-939c-5d1bafda539d" containerName="main"
Sep 09 21:23:29.583841 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.583820    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.587855 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.587833    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-jxfn7\""
Sep 09 21:23:29.587855 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.587850    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Sep 09 21:23:29.588005 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.587833    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Sep 09 21:23:29.588005 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.587832    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Sep 09 21:23:29.594290 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.594267    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"]
Sep 09 21:23:29.625933 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.625905    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:29.708609 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.708582    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm8jm\" (UniqueName: \"kubernetes.io/projected/34c6caee-ceef-4b8e-b992-d3a801c653ac-kube-api-access-lm8jm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.708609 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.708611    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.708776 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.708626    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.708776 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.708647    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c6caee-ceef-4b8e-b992-d3a801c653ac-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.708776 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.708746    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.708898 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.708802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810099 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810075    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810237 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810118    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810237 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810182    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm8jm\" (UniqueName: \"kubernetes.io/projected/34c6caee-ceef-4b8e-b992-d3a801c653ac-kube-api-access-lm8jm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810237 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810209    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810237 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810482 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810284    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c6caee-ceef-4b8e-b992-d3a801c653ac-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810542 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810613 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810593    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.810613 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.810598    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.817913 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.817886    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.818480 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.818463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c6caee-ceef-4b8e-b992-d3a801c653ac-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.821274 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.821232    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm8jm\" (UniqueName: \"kubernetes.io/projected/34c6caee-ceef-4b8e-b992-d3a801c653ac-kube-api-access-lm8jm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-k9n9q\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.883815 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.883783    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:29.894215 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.894188    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:29.981930 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:29.981906    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:23:30.113029 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.112961    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-token\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113029 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113000    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-credential-socket\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113057    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-data\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113081    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks4m5\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-kube-api-access-ks4m5\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113103    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3045579-a8ca-4093-ab53-e27ca10bba9a-istiod-ca-cert\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113127    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-podinfo\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113153    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-socket\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113204    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-envoy\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113569 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113232    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-certs\") pod \"f3045579-a8ca-4093-ab53-e27ca10bba9a\" (UID: \"f3045579-a8ca-4093-ab53-e27ca10bba9a\") "
Sep 09 21:23:30.113569 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113409    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:23:30.113569 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113491    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:23:30.113569 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113553    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:23:30.113774 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113570    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-data\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.113774 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113698    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:23:30.113774 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.113724    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3045579-a8ca-4093-ab53-e27ca10bba9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:23:30.115226 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.115203    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:23:30.115359 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.115262    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-kube-api-access-ks4m5" (OuterVolumeSpecName: "kube-api-access-ks4m5") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "kube-api-access-ks4m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:23:30.115519 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.115498    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:23:30.115580 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.115551    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3045579-a8ca-4093-ab53-e27ca10bba9a" (UID: "f3045579-a8ca-4093-ab53-e27ca10bba9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:23:30.177806 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.177787    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"]
Sep 09 21:23:30.179757 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:23:30.179735    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c6caee_ceef_4b8e_b992_d3a801c653ac.slice/crio-2ef5f095730c4144a130992cc9505c63ba2cf8b024369195acbbf2a7ac230d21 WatchSource:0}: Error finding container 2ef5f095730c4144a130992cc9505c63ba2cf8b024369195acbbf2a7ac230d21: Status 404 returned error can't find the container with id 2ef5f095730c4144a130992cc9505c63ba2cf8b024369195acbbf2a7ac230d21
Sep 09 21:23:30.214134 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214115    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-envoy\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214134 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214134    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214142    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-token\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214151    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-credential-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214162    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks4m5\" (UniqueName: \"kubernetes.io/projected/f3045579-a8ca-4093-ab53-e27ca10bba9a-kube-api-access-ks4m5\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214174    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3045579-a8ca-4093-ab53-e27ca10bba9a-istiod-ca-cert\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214185    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3045579-a8ca-4093-ab53-e27ca10bba9a-istio-podinfo\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.214287 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.214195    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3045579-a8ca-4093-ab53-e27ca10bba9a-workload-socket\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:23:30.814904 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.814868    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" event={"ID":"34c6caee-ceef-4b8e-b992-d3a801c653ac","Type":"ContainerStarted","Data":"009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"}
Sep 09 21:23:30.815356 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.814931    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" event={"ID":"34c6caee-ceef-4b8e-b992-d3a801c653ac","Type":"ContainerStarted","Data":"2ef5f095730c4144a130992cc9505c63ba2cf8b024369195acbbf2a7ac230d21"}
Sep 09 21:23:30.816432 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.816408    2573 generic.go:356] "Generic (PLEG): container finished" podID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerID="228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae" exitCode=0
Sep 09 21:23:30.816554 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.816448    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"
Sep 09 21:23:30.816554 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.816479    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" event={"ID":"f3045579-a8ca-4093-ab53-e27ca10bba9a","Type":"ContainerDied","Data":"228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae"}
Sep 09 21:23:30.816554 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.816526    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg" event={"ID":"f3045579-a8ca-4093-ab53-e27ca10bba9a","Type":"ContainerDied","Data":"4132475a9a944e02b9b9a19187d7d67dce029a7dab92f6a8676574455a900169"}
Sep 09 21:23:30.816554 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.816552    2573 scope.go:117] "RemoveContainer" containerID="228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae"
Sep 09 21:23:30.824938 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.824902    2573 scope.go:117] "RemoveContainer" containerID="5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"
Sep 09 21:23:30.831983 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.831966    2573 scope.go:117] "RemoveContainer" containerID="228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae"
Sep 09 21:23:30.832224 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:23:30.832204    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae\": container with ID starting with 228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae not found: ID does not exist" containerID="228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae"
Sep 09 21:23:30.832317 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.832231    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae"} err="failed to get container status \"228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae\": rpc error: code = NotFound desc = could not find container \"228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae\": container with ID starting with 228640a3e930d40bca323d0adc2a2fea5851e3d1fb3625577d81653bcc45beae not found: ID does not exist"
Sep 09 21:23:30.832317 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.832246    2573 scope.go:117] "RemoveContainer" containerID="5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"
Sep 09 21:23:30.832519 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:23:30.832504    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7\": container with ID starting with 5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7 not found: ID does not exist" containerID="5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"
Sep 09 21:23:30.832569 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.832522    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7"} err="failed to get container status \"5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7\": rpc error: code = NotFound desc = could not find container \"5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7\": container with ID starting with 5e3eea58afa20610ffc70e96ec25dae9abbc61ae15c7d84695c333a70e60a8c7 not found: ID does not exist"
Sep 09 21:23:30.833792 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.833752    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" podStartSLOduration=1.833741071 podStartE2EDuration="1.833741071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:23:30.833230785 +0000 UTC m=+4349.764314115" watchObservedRunningTime="2026-09-09 21:23:30.833741071 +0000 UTC m=+4349.764824425"
Sep 09 21:23:30.853777 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.853755    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"]
Sep 09 21:23:30.860617 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.860582    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-5858dff66d-k6fqg"]
Sep 09 21:23:30.894748 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:30.894726    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:31.641425 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:31.641398    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" path="/var/lib/kubelet/pods/f3045579-a8ca-4093-ab53-e27ca10bba9a/volumes"
Sep 09 21:23:36.469760 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:36.469720    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:38.685600 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:38.685567    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:39.895056 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:39.895017    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:39.907041 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:39.907017    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:40.872798 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:40.872770    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:23:46.306437 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:46.306401    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:48.240427 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:48.240398    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:54.302814 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:23:54.302773    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:24:02.586898 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:02.586862    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:24:09.799951 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:09.799916    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:24:09.903921 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:09.898385    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:24:13.953470 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:13.953431    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"]
Sep 09 21:24:13.962131 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:24:13.962084    2573 log.go:32] "ExecSync cmd from runtime service failed" err=<
Sep 09 21:24:13.962131 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 21:24:13.962131 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 21:24:13.962131 ip-10-0-132-68 kubenswrapper[2573]:  > containerID="009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6" cmd=["/bin/sleep","15"]
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:13.962147    2573 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" message="" err=<
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]:  >
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:24:13.962160    2573 kuberuntime_container.go:755] "PreStop hook failed" err=<
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]:         , stdout: , stderr: , exit code -1
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" podUID="34c6caee-ceef-4b8e-b992-d3a801c653ac" containerName="main" containerID="cri-o://009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"
Sep 09 21:24:13.962485 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:13.962208    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" podUID="34c6caee-ceef-4b8e-b992-d3a801c653ac" containerName="main" containerID="cri-o://009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6" gracePeriod=60
Sep 09 21:24:14.698822 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.698798    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:24:14.783216 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783186    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-tmp-dir\") pod \"34c6caee-ceef-4b8e-b992-d3a801c653ac\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") "
Sep 09 21:24:14.783216 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783219    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-dshm\") pod \"34c6caee-ceef-4b8e-b992-d3a801c653ac\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") "
Sep 09 21:24:14.783455 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783299    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-home\") pod \"34c6caee-ceef-4b8e-b992-d3a801c653ac\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") "
Sep 09 21:24:14.783455 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783348    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-model-cache\") pod \"34c6caee-ceef-4b8e-b992-d3a801c653ac\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") "
Sep 09 21:24:14.783455 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783377    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c6caee-ceef-4b8e-b992-d3a801c653ac-tls-certs\") pod \"34c6caee-ceef-4b8e-b992-d3a801c653ac\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") "
Sep 09 21:24:14.783455 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783421    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm8jm\" (UniqueName: \"kubernetes.io/projected/34c6caee-ceef-4b8e-b992-d3a801c653ac-kube-api-access-lm8jm\") pod \"34c6caee-ceef-4b8e-b992-d3a801c653ac\" (UID: \"34c6caee-ceef-4b8e-b992-d3a801c653ac\") "
Sep 09 21:24:14.783636 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783550    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "34c6caee-ceef-4b8e-b992-d3a801c653ac" (UID: "34c6caee-ceef-4b8e-b992-d3a801c653ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:24:14.783636 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783623    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-home" (OuterVolumeSpecName: "home") pod "34c6caee-ceef-4b8e-b992-d3a801c653ac" (UID: "34c6caee-ceef-4b8e-b992-d3a801c653ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:24:14.783752 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783609    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "34c6caee-ceef-4b8e-b992-d3a801c653ac" (UID: "34c6caee-ceef-4b8e-b992-d3a801c653ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:24:14.783752 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783697    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-home\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:24:14.783752 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783713    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-model-cache\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:24:14.783752 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.783727    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-tmp-dir\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:24:14.785649 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.785622    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-dshm" (OuterVolumeSpecName: "dshm") pod "34c6caee-ceef-4b8e-b992-d3a801c653ac" (UID: "34c6caee-ceef-4b8e-b992-d3a801c653ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:24:14.785763 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.785646    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c6caee-ceef-4b8e-b992-d3a801c653ac-kube-api-access-lm8jm" (OuterVolumeSpecName: "kube-api-access-lm8jm") pod "34c6caee-ceef-4b8e-b992-d3a801c653ac" (UID: "34c6caee-ceef-4b8e-b992-d3a801c653ac"). InnerVolumeSpecName "kube-api-access-lm8jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:24:14.785763 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.785656    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34c6caee-ceef-4b8e-b992-d3a801c653ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "34c6caee-ceef-4b8e-b992-d3a801c653ac" (UID: "34c6caee-ceef-4b8e-b992-d3a801c653ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:24:14.885019 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.884991    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c6caee-ceef-4b8e-b992-d3a801c653ac-tls-certs\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:24:14.885019 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.885022    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm8jm\" (UniqueName: \"kubernetes.io/projected/34c6caee-ceef-4b8e-b992-d3a801c653ac-kube-api-access-lm8jm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:24:14.885219 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:14.885036    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c6caee-ceef-4b8e-b992-d3a801c653ac-dshm\") on node \"ip-10-0-132-68.ec2.internal\" DevicePath \"\""
Sep 09 21:24:15.017437 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.017402    2573 generic.go:356] "Generic (PLEG): container finished" podID="34c6caee-ceef-4b8e-b992-d3a801c653ac" containerID="009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6" exitCode=0
Sep 09 21:24:15.017913 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.017447    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" event={"ID":"34c6caee-ceef-4b8e-b992-d3a801c653ac","Type":"ContainerDied","Data":"009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"}
Sep 09 21:24:15.017913 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.017458    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"
Sep 09 21:24:15.017913 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.017486    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q" event={"ID":"34c6caee-ceef-4b8e-b992-d3a801c653ac","Type":"ContainerDied","Data":"2ef5f095730c4144a130992cc9505c63ba2cf8b024369195acbbf2a7ac230d21"}
Sep 09 21:24:15.017913 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.017507    2573 scope.go:117] "RemoveContainer" containerID="009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"
Sep 09 21:24:15.026043 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.026023    2573 scope.go:117] "RemoveContainer" containerID="009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"
Sep 09 21:24:15.026322 ip-10-0-132-68 kubenswrapper[2573]: E0909 21:24:15.026304    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6\": container with ID starting with 009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6 not found: ID does not exist" containerID="009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"
Sep 09 21:24:15.026375 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.026330    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6"} err="failed to get container status \"009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6\": rpc error: code = NotFound desc = could not find container \"009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6\": container with ID starting with 009e4bbe6eac1c02ea1c2c489b289b7f4225465445b8cf74fe09124f1917b9d6 not found: ID does not exist"
Sep 09 21:24:15.044296 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.044273    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"]
Sep 09 21:24:15.049620 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.049596    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-k9n9q"]
Sep 09 21:24:15.641552 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:15.641510    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c6caee-ceef-4b8e-b992-d3a801c653ac" path="/var/lib/kubelet/pods/34c6caee-ceef-4b8e-b992-d3a801c653ac/volumes"
Sep 09 21:24:21.602886 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:21.602848    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:24:21.776415 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:24:21.776382    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:06.327289 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:06.327241    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:06.579878 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:06.579792    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:10.392151 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:10.392113    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:17.335691 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:17.335658    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:23.388417 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:23.388383    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:23.488684 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:23.488651    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:53.198603 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:53.198568    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:56.508540 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:56.508458    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:56.585237 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:56.585209    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:59.854119 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:25:59.854091    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:01.839297 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:01.839269    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:26:01.854809 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:01.854779    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:26:01.857076 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:01.857054    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:26:01.865928 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:01.865907    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2frcb_98cddd01-8d4d-45e7-86d0-f8edb8d38c1a/ovn-acl-logging/0.log"
Sep 09 21:26:05.829245 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:05.829214    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:05.970359 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:05.970327    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:22.003687 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:22.003654    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6fc8f864f4-mjx5g_805aa70d-a43f-47c7-a9be-58372b88cc06/manager/0.log"
Sep 09 21:26:22.185623 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:22.185593    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-2q7lb_8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3/limitador/0.log"
Sep 09 21:26:28.125178 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:28.125146    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:28.290407 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:28.290375    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:28.391785 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:28.391708    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:31.316120 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:31.316069    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h49zk_41cf389f-73c5-4623-ba64-c2efc1782d75/global-pull-secret-syncer/0.log"
Sep 09 21:26:31.420416 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:31.420386    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log"
Sep 09 21:26:31.562153 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:31.562127    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-68.ec2.internal_80384244c83dfd7f47c7ded897b4c708/haproxy/0.log"
Sep 09 21:26:34.675132 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:34.675101    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:34.750839 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:34.750807    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:36.673986 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:36.673956    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6fc8f864f4-mjx5g_805aa70d-a43f-47c7-a9be-58372b88cc06/manager/0.log"
Sep 09 21:26:36.968299 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:36.968204    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-2q7lb_8f41ad76-b2f3-4b74-b6a8-18c06fc4a8a3/limitador/0.log"
Sep 09 21:26:38.384317 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.384283    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-9p225_7d96ce2f-0d16-4c53-a1f0-9b4e5d363441/cluster-monitoring-operator/0.log"
Sep 09 21:26:38.509597 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.509554    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6646dfc98-csdnq_ecf9620c-dd13-4c5a-a61b-04bd353c9877/metrics-server/0.log"
Sep 09 21:26:38.538310 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.538293    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-bnlxx_48e41d1b-87df-49f6-9908-e2bc20f8210e/monitoring-plugin/0.log"
Sep 09 21:26:38.785607 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.785580    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l87wp_638e9f45-54c2-4de4-bdd6-ca54251673de/node-exporter/0.log"
Sep 09 21:26:38.809617 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.809598    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l87wp_638e9f45-54c2-4de4-bdd6-ca54251673de/kube-rbac-proxy/0.log"
Sep 09 21:26:38.836545 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.836526    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l87wp_638e9f45-54c2-4de4-bdd6-ca54251673de/init-textfile/0.log"
Sep 09 21:26:38.977159 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.977094    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/0.log"
Sep 09 21:26:38.982090 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:38.982067    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/prometheus/1.log"
Sep 09 21:26:39.008407 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.008384    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/config-reloader/0.log"
Sep 09 21:26:39.035281 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.035235    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/thanos-sidecar/0.log"
Sep 09 21:26:39.060498 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.060483    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/kube-rbac-proxy-web/0.log"
Sep 09 21:26:39.087011 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.086992    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/kube-rbac-proxy/0.log"
Sep 09 21:26:39.118325 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.118302    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/kube-rbac-proxy-thanos/0.log"
Sep 09 21:26:39.149352 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.149333    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2279281c-31c4-460d-80da-0f9a4cae52ce/init-config-reloader/0.log"
Sep 09 21:26:39.186144 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.186122    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pq466_d3f2aed8-212b-4623-bd72-218e131a26c3/prometheus-operator/0.log"
Sep 09 21:26:39.209386 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.209358    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pq466_d3f2aed8-212b-4623-bd72-218e131a26c3/kube-rbac-proxy/0.log"
Sep 09 21:26:39.240193 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.240133    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-69xvj_6d9a0de5-5313-4073-8fe3-dac62b357b1d/prometheus-operator-admission-webhook/0.log"
Sep 09 21:26:39.382735 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.382714    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55797c89d5-tz4p5_f0e42510-ddf9-40ce-914a-db8ad52e574a/thanos-query/0.log"
Sep 09 21:26:39.427384 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.427363    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55797c89d5-tz4p5_f0e42510-ddf9-40ce-914a-db8ad52e574a/kube-rbac-proxy-web/0.log"
Sep 09 21:26:39.484459 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.484439    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55797c89d5-tz4p5_f0e42510-ddf9-40ce-914a-db8ad52e574a/kube-rbac-proxy/0.log"
Sep 09 21:26:39.536111 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.536047    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55797c89d5-tz4p5_f0e42510-ddf9-40ce-914a-db8ad52e574a/prom-label-proxy/0.log"
Sep 09 21:26:39.569628 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.569600    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55797c89d5-tz4p5_f0e42510-ddf9-40ce-914a-db8ad52e574a/kube-rbac-proxy-rules/0.log"
Sep 09 21:26:39.595914 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:39.595874    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55797c89d5-tz4p5_f0e42510-ddf9-40ce-914a-db8ad52e574a/kube-rbac-proxy-metrics/0.log"
Sep 09 21:26:40.724267 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:40.724217    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-69ggj_21f2a60f-d391-409c-a92f-a4b3ea9b9c9b/networking-console-plugin/0.log"
Sep 09 21:26:41.177494 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177458    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"]
Sep 09 21:26:41.177778 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177765    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6caee-ceef-4b8e-b992-d3a801c653ac" containerName="main"
Sep 09 21:26:41.177826 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177780    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6caee-ceef-4b8e-b992-d3a801c653ac" containerName="main"
Sep 09 21:26:41.177826 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177791    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy"
Sep 09 21:26:41.177826 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177797    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy"
Sep 09 21:26:41.177919 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177859    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3045579-a8ca-4093-ab53-e27ca10bba9a" containerName="istio-proxy"
Sep 09 21:26:41.177919 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.177868    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6caee-ceef-4b8e-b992-d3a801c653ac" containerName="main"
Sep 09 21:26:41.181138 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.181118    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.183778 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.183758    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sghw5\"/\"openshift-service-ca.crt\""
Sep 09 21:26:41.183898 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.183793    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sghw5\"/\"kube-root-ca.crt\""
Sep 09 21:26:41.184706 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.184690    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sghw5\"/\"default-dockercfg-mcj2p\""
Sep 09 21:26:41.189049 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.189030    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"]
Sep 09 21:26:41.328833 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.328801    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-podres\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.328833 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.328836    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfpwt\" (UniqueName: \"kubernetes.io/projected/38922c7f-2120-4ce3-8eac-f1591ed312e4-kube-api-access-rfpwt\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.329068 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.328865    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-proc\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.329068 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.328947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-lib-modules\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.329068 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.328983    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-sys\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429563 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429495    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-proc\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429563 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429528    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-lib-modules\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429563 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-sys\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429597    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-podres\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429615    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfpwt\" (UniqueName: \"kubernetes.io/projected/38922c7f-2120-4ce3-8eac-f1591ed312e4-kube-api-access-rfpwt\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429616    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-proc\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429666    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-sys\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-lib-modules\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.429799 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.429757    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38922c7f-2120-4ce3-8eac-f1591ed312e4-podres\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.438612 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.438591    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfpwt\" (UniqueName: \"kubernetes.io/projected/38922c7f-2120-4ce3-8eac-f1591ed312e4-kube-api-access-rfpwt\") pod \"perf-node-gather-daemonset-fcjbm\" (UID: \"38922c7f-2120-4ce3-8eac-f1591ed312e4\") " pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.491670 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.491639    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:41.762220 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:41.762192    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"]
Sep 09 21:26:41.764860 ip-10-0-132-68 kubenswrapper[2573]: W0909 21:26:41.764833    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod38922c7f_2120_4ce3_8eac_f1591ed312e4.slice/crio-816eacd2be4aecafc14d274ca406275e239573f22a688830087e2fe452c50f6c WatchSource:0}: Error finding container 816eacd2be4aecafc14d274ca406275e239573f22a688830087e2fe452c50f6c: Status 404 returned error can't find the container with id 816eacd2be4aecafc14d274ca406275e239573f22a688830087e2fe452c50f6c
Sep 09 21:26:42.656830 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:42.656792    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm" event={"ID":"38922c7f-2120-4ce3-8eac-f1591ed312e4","Type":"ContainerStarted","Data":"780f3c399740e40f926d506acb20165d4caa15478b75a17c5b0e7b00195684b6"}
Sep 09 21:26:42.657023 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:42.656962    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:42.657023 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:42.657006    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm" event={"ID":"38922c7f-2120-4ce3-8eac-f1591ed312e4","Type":"ContainerStarted","Data":"816eacd2be4aecafc14d274ca406275e239573f22a688830087e2fe452c50f6c"}
Sep 09 21:26:42.675265 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:42.675206    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm" podStartSLOduration=1.675191026 podStartE2EDuration="1.675191026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:26:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:42.674179174 +0000 UTC m=+4541.605262492" watchObservedRunningTime="2026-09-09 21:26:42.675191026 +0000 UTC m=+4541.606274343"
Sep 09 21:26:43.460126 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:43.460094    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkb4d_bdee7bc4-c3c5-4ee2-9913-f967f8649d42/dns/0.log"
Sep 09 21:26:43.484600 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:43.484576    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkb4d_bdee7bc4-c3c5-4ee2-9913-f967f8649d42/kube-rbac-proxy/0.log"
Sep 09 21:26:43.619206 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:43.619173    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jfvgh_aeecf69a-3522-49fa-924f-3208e1e1af06/dns-node-resolver/0.log"
Sep 09 21:26:44.324068 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:44.323833    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xf98s_e8e60aba-2480-4462-ac8b-15a1a89661fd/node-ca/0.log"
Sep 09 21:26:46.251792 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:46.251763    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rr9dx_e682d8e2-2fc0-468d-8db5-1e1f6372a43c/serve-healthcheck-canary/0.log"
Sep 09 21:26:46.709481 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:46.709451    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dm4lq_0dd432c4-c58f-4523-a3a0-6ca0b6e27366/insights-operator/0.log"
Sep 09 21:26:46.710064 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:46.710049    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-dm4lq_0dd432c4-c58f-4523-a3a0-6ca0b6e27366/insights-operator/1.log"
Sep 09 21:26:46.915011 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:46.914986    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gtv5p_0de91534-394e-41b0-b58f-a199613bc5f6/kube-rbac-proxy/0.log"
Sep 09 21:26:46.942486 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:46.942463    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gtv5p_0de91534-394e-41b0-b58f-a199613bc5f6/exporter/0.log"
Sep 09 21:26:46.966933 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:46.966869    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gtv5p_0de91534-394e-41b0-b58f-a199613bc5f6/extractor/0.log"
Sep 09 21:26:48.670934 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:48.670903    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sghw5/perf-node-gather-daemonset-fcjbm"
Sep 09 21:26:49.029309 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:49.029219    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:49.134969 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:49.134936    2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9jmtd_9063d1b5-2af6-4334-a199-9addeb600283/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:50.099003 ip-10-0-132-68 kubenswrapper[2573]: I0909 21:26:50.098970    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f7677c75-8nq6x_d6d40538-f779-4dad-b8a9-3b0f50f6fe31/manager/0.log"