Aug 05 17:12:21.355604 ip-10-0-131-218 systemd[1]: Starting Kubernetes Kubelet...
Aug 05 17:12:21.739129 ip-10-0-131-218 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:12:21.739129 ip-10-0-131-218 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 05 17:12:21.739129 ip-10-0-131-218 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:12:21.739129 ip-10-0-131-218 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 05 17:12:21.739129 ip-10-0-131-218 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:12:21.741473 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.741388    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 05 17:12:21.748102 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748086    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:21.748102 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748101    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748105    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748109    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748112    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748115    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748118    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748121    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748124    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748127    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748130    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748133    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748136    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748139    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748142    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748145    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748147    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748152    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748156    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748159    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:21.748169 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748162    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748166    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748170    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748173    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748176    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748179    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748183    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748186    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748189    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748191    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748194    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748197    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748199    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748202    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748204    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748207    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748209    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748212    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748215    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:21.748636 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748217    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748220    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748223    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748225    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748229    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748231    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748234    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748236    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748239    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748241    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748244    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748246    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748249    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748251    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748254    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748257    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748260    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748263    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748265    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748268    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:21.749095 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748271    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748273    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748276    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748278    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748281    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748283    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748286    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748289    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748291    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748294    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748297    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748299    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748303    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748306    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748309    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748322    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748325    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748328    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748330    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748333    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:21.749681 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748335    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748338    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748341    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748344    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748347    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748349    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748352    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748355    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748358    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748361    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748363    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748366    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748369    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748371    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748374    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748376    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748379    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748382    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748386    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:21.750161 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.748389    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749074    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749082    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749085    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749088    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749091    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749094    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749097    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749100    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749103    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749106    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749109    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749111    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749114    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749117    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749119    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749122    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749124    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749127    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:21.750643 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749129    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749132    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749135    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749138    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749140    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749143    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749146    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749148    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749151    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749154    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749157    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749159    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749162    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749164    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749167    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749169    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749172    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749174    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749177    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749181    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:21.751118 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749185    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749189    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749193    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749196    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749199    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749201    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749204    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749206    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749209    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749212    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749215    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749218    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749220    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749223    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749226    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749228    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749231    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749234    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749236    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749240    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:21.751613 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749243    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749245    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749248    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749251    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749253    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749255    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749258    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749261    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749263    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749266    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749268    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749271    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749273    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749276    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749279    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749281    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749283    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749286    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749288    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749291    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:21.752096 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749293    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749296    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749299    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749301    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749304    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749307    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749323    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749327    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749329    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749332    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749334    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749337    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749340    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749342    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749345    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749348    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749350    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749353    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749356    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749358    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:21.752611 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749361    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749728    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749733    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749736    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749739    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749742    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749745    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749747    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749750    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749753    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749756    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749759    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749761    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749764    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749767    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749771    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749773    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749776    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749779    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749782    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:21.753105 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749784    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749787    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749790    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749793    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749795    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749798    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749800    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749803    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749806    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749808    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749811    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749813    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749816    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749819    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749821    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749824    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749826    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749829    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749832    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749835    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:21.753601 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749838    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749841    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749843    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749845    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749848    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749851    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749853    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749857    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749860    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749863    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749866    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749868    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749871    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749873    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749876    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749879    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749881    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749884    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749886    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:21.754085 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749889    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749891    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749894    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749897    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749900    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749902    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749904    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749907    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749909    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749912    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749914    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749917    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749920    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749922    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749925    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749929    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749932    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749936    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749939    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:21.754600 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749942    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749944    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749947    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749950    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749952    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749955    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749959    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749962    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749965    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749968    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749970    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749973    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749976    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749978    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749981    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749984    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749987    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749989    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749992    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749994    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:21.755071 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749997    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.749999    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750069    2562 flags.go:64] FLAG: --address="0.0.0.0"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750075    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750082    2562 flags.go:64] FLAG: --anonymous-auth="true"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750086    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750091    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750094    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750099    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750103    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750106    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750110    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750113    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750116    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750120    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750122    2562 flags.go:64] FLAG: --cgroup-root=""
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750125    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750128    2562 flags.go:64] FLAG: --client-ca-file=""
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750131    2562 flags.go:64] FLAG: --cloud-provider="external"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750134    2562 flags.go:64] FLAG: --cluster-dns="[]"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750138    2562 flags.go:64] FLAG: --cluster-domain=""
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750141    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750144    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750147    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 05 17:12:21.755560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750151    2562 flags.go:64] FLAG: --container-log-max-files="5"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750154    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750157    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750160    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750164    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750167    2562 flags.go:64] FLAG: --contention-profiling="false"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750170    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750173    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750177    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750179    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750183    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750187    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750190    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750192    2562 flags.go:64] FLAG: --enable-load-reader="false"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750195    2562 flags.go:64] FLAG: --enable-server="true"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750199    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750203    2562 flags.go:64] FLAG: --event-burst="100"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750207    2562 flags.go:64] FLAG: --event-qps="50"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750212    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750215    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750218    2562 flags.go:64] FLAG: --eviction-hard=""
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750222    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750225    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750228    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750231    2562 flags.go:64] FLAG: --eviction-soft=""
Aug 05 17:12:21.756175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750234    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750237    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750239    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750242    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750245    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750248    2562 flags.go:64] FLAG: --fail-swap-on="true"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750250    2562 flags.go:64] FLAG: --feature-gates=""
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750254    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750257    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750260    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750263    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750266    2562 flags.go:64] FLAG: --healthz-port="10248"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750269    2562 flags.go:64] FLAG: --help="false"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750272    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-131-218.ec2.internal"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750275    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750278    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750281    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750284    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750288    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750291    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750294    2562 flags.go:64] FLAG: --image-service-endpoint=""
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750297    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750300    2562 flags.go:64] FLAG: --kube-api-burst="100"
Aug 05 17:12:21.756796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750303    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750306    2562 flags.go:64] FLAG: --kube-api-qps="50"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750324    2562 flags.go:64] FLAG: --kube-reserved=""
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750327    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750330    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750333    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750336    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750339    2562 flags.go:64] FLAG: --lock-file=""
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750342    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750345    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750348    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750353    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750356    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750359    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750361    2562 flags.go:64] FLAG: --logging-format="text"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750364    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750368    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750370    2562 flags.go:64] FLAG: --manifest-url=""
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750373    2562 flags.go:64] FLAG: --manifest-url-header=""
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750377    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750380    2562 flags.go:64] FLAG: --max-open-files="1000000"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750384    2562 flags.go:64] FLAG: --max-pods="110"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750387    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750390    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750393    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 05 17:12:21.757391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750396    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750399    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750402    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750405    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750414    2562 flags.go:64] FLAG: --node-status-max-images="50"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750417    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750419    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750423    2562 flags.go:64] FLAG: --pod-cidr=""
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750426    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750431    2562 flags.go:64] FLAG: --pod-manifest-path=""
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750435    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750439    2562 flags.go:64] FLAG: --pods-per-core="0"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750442    2562 flags.go:64] FLAG: --port="10250"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750445    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750448    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e9370e4fcee67223"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750451    2562 flags.go:64] FLAG: --qos-reserved=""
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750454    2562 flags.go:64] FLAG: --read-only-port="10255"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750457    2562 flags.go:64] FLAG: --register-node="true"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750459    2562 flags.go:64] FLAG: --register-with-taints=""
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750463    2562 flags.go:64] FLAG: --registry-burst="10"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750466    2562 flags.go:64] FLAG: --registry-qps="5"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750469    2562 flags.go:64] FLAG: --reserved-cpus=""
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750472    2562 flags.go:64] FLAG: --reserved-memory=""
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750475    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750478    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 05 17:12:21.758007 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750481    2562 flags.go:64] FLAG: --rotate-certificates="false"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750484    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750487    2562 flags.go:64] FLAG: --runonce="false"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750490    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750493    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750496    2562 flags.go:64] FLAG: --seccomp-default="false"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750498    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750501    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750505    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750508    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750511    2562 flags.go:64] FLAG: --storage-driver-password="root"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750515    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750518    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750521    2562 flags.go:64] FLAG: --storage-driver-user="root"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750524    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750526    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750530    2562 flags.go:64] FLAG: --system-cgroups=""
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750534    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750539    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750542    2562 flags.go:64] FLAG: --tls-cert-file=""
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750545    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750548    2562 flags.go:64] FLAG: --tls-min-version=""
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750551    2562 flags.go:64] FLAG: --tls-private-key-file=""
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750554    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750557    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 05 17:12:21.758640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750560    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750562    2562 flags.go:64] FLAG: --v="2"
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750567    2562 flags.go:64] FLAG: --version="false"
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750571    2562 flags.go:64] FLAG: --vmodule=""
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750574    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750577    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750695    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750700    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750705    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750708    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750711    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750714    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750716    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750719    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750722    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750725    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750727    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750730    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750736    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750739    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:21.759265 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750743    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750746    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750749    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750752    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750763    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750766    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750768    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750771    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750774    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750776    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750779    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750782    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750786    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750789    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750792    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750795    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750798    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750801    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750803    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:21.759825 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750806    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750809    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750811    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750814    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750816    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750818    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750821    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750824    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750826    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750829    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750832    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750836    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750838    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750841    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750844    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750846    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750849    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750853    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750856    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750858    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:21.760283 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750861    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750864    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750867    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750869    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750872    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750874    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750877    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750879    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750882    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750884    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750887    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750889    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750892    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750894    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750897    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750899    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750902    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750904    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750907    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750910    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:21.760795 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750912    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750915    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750917    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750921    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750924    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750926    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750929    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750932    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750934    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750938    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750941    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750944    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750946    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750949    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750952    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750955    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750957    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750960    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750962    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750965    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:21.761301 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750967    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750970    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750973    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750975    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750978    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.750980    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.750985    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.757902    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.757920    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.757938    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.757948    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758010    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758015    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758020    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:21.761836 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758024    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758027    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758030    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758033    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758036    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758038    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758041    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758045    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758049    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758051    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758055    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758058    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758061    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758064    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758066    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758069    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758072    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758075    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758077    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:21.762182 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758080    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758083    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758086    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758090    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758093    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758096    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758098    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758108    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758111    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758113    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758116    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758118    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758121    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758124    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758126    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758129    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758132    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758134    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758137    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:21.762763 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758140    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758143    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758146    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758148    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758150    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758153    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758156    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758159    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758162    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758164    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758167    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758169    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758172    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758175    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758177    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758180    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758182    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758185    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758188    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758190    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:21.763237 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758193    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758201    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758204    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758207    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758210    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758212    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758215    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758217    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758220    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758222    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758225    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758227    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758229    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758232    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758234    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758237    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758239    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758242    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758245    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758248    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:21.763744 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758250    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758253    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758255    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758257    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758260    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758263    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758265    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758268    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758271    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758273    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758276    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758278    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758281    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758284    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758292    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758295    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758297    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:21.764370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758300    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.758305    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758468    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758473    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758476    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758479    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758482    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758485    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758487    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758490    2562 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758493    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758495    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758498    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758501    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758504    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:21.764950 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758506    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758509    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758512    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758515    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758517    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758520    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758522    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758526    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758529    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758532    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758534    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758537    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758539    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758542    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758544    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758554    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758557    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758560    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758563    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:21.765496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758566    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758568    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758571    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758573    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758576    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758578    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758581    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758584    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758587    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758590    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758593    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758596    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758598    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758601    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758603    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758606    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758608    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758611    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758614    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:21.766045 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758616    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758619    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758621    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758624    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758626    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758643    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758647    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758649    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758652    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758655    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758665    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758668    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758670    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758673    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758675    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758678    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758681    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758683    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758687    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758691    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:21.766558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758693    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758696    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758699    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758702    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758704    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758707    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758710    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758712    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758715    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758718    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758720    2562 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758723    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758725    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758728    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758730    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758733    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758744    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758747    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758749    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758752    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:21.767064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758755    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758757    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758760    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758768    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758771    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758774    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758776    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:21.758778    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.758783    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.758921    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.760975    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.762419    2562 server.go:1019] "Starting client certificate rotation"
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.762547    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.762586    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:12:21.767559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.767496    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:12:21.772825 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.772808    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:12:21.781720 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.781704    2562 log.go:25] "Validated CRI v1 runtime API"
Aug 05 17:12:21.783704 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.783690    2562 log.go:25] "Validated CRI v1 image API"
Aug 05 17:12:21.785054 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.785040    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 05 17:12:21.790553 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.790533    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7cca2ed2-9d33-4427-8f43-b0a442d6f73b:/dev/nvme0n1p3 f8c8335d-e8cf-42ad-bf1b-0db6bb781d7c:/dev/nvme0n1p4]
Aug 05 17:12:21.790611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.790551    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 05 17:12:21.794681 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.794664    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:12:21.795805 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.795698    2562 manager.go:216] Machine: {Timestamp:2026-08-05 17:12:21.794514389 +0000 UTC m=+0.327698624 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097775 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24f7749dcb5cefde28db58d9bdefe9 SystemUUID:ec24f774-9dcb-5cef-de28-db58d9bdefe9 BootID:a3c6141f-39e1-4a27-8a43-174006a57cdc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:75:bb:52:2b:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:75:bb:52:2b:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:1c:31:25:0c:3c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 05 17:12:21.795805 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.795805    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 05 17:12:21.795909 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.795880    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 05 17:12:21.798230 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.798205    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 05 17:12:21.798378 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.798233    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-218.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 05 17:12:21.799033 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.799023    2562 topology_manager.go:138] "Creating topology manager with none policy"
Aug 05 17:12:21.799067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.799036    2562 container_manager_linux.go:309] "Creating device plugin manager"
Aug 05 17:12:21.799067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.799047    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:12:21.799676 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.799666    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:12:21.800790 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.800781    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 05 17:12:21.803880 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.803870    2562 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:12:21.803982 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.803973    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 05 17:12:21.804015 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804000    2562 kubelet.go:486] "Attempting to sync node with API server"
Aug 05 17:12:21.804015 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804009    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 05 17:12:21.804064 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804022    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 05 17:12:21.804064 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804032    2562 kubelet.go:398] "Adding apiserver pod source"
Aug 05 17:12:21.804064 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804040    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 05 17:12:21.804438 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804427    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:12:21.804487 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.804444    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:12:21.808977 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.808402    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 05 17:12:21.810217 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810199    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 05 17:12:21.810303 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810240    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 05 17:12:21.810879 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810863    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 05 17:12:21.810941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810886    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 05 17:12:21.810941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810896    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 05 17:12:21.810941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810905    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 05 17:12:21.810941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810917    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 05 17:12:21.810941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810926    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810977    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810987    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.810997    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.811005    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.811025    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.811038    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.811067    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 05 17:12:21.811165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.811075    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 05 17:12:21.814072 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.814053    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-218.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 05 17:12:21.814358 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.814345    2562 server.go:1268] "Started kubelet"
Aug 05 17:12:21.814449 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.814421    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 05 17:12:21.814575 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.814552    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 05 17:12:21.814672 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.814588    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 05 17:12:21.814782 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.814694    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 05 17:12:21.814851 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.814822    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 05 17:12:21.815185 ip-10-0-131-218 systemd[1]: Started Kubernetes Kubelet.
Aug 05 17:12:21.816248 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.815880    2562 server.go:310] "Adding debug handlers to kubelet server"
Aug 05 17:12:21.816451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.816435    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 05 17:12:21.820293 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.820266    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 05 17:12:21.822168 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822151    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 05 17:12:21.822409 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822153    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 05 17:12:21.822409 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.822256    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:21.822786 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822689    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 05 17:12:21.822786 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822693    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 05 17:12:21.822786 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822717    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 05 17:12:21.822958 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822811    2562 reconstruct.go:97] "Volume reconstruction finished"
Aug 05 17:12:21.822958 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.822819    2562 reconciler.go:29] "Reconciler: start to sync state"
Aug 05 17:12:21.823108 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823091    2562 factory.go:153] Registering CRI-O factory
Aug 05 17:12:21.823108 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823108    2562 factory.go:223] Registration of the crio container factory successfully
Aug 05 17:12:21.823229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823157    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 05 17:12:21.823229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823179    2562 factory.go:55] Registering systemd factory
Aug 05 17:12:21.823229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823187    2562 factory.go:223] Registration of the systemd container factory successfully
Aug 05 17:12:21.823229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823206    2562 factory.go:103] Registering Raw factory
Aug 05 17:12:21.823229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823219    2562 manager.go:1196] Started watching for new ooms in manager
Aug 05 17:12:21.823663 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.823649    2562 manager.go:349] Starting recovery of all containers
Aug 05 17:12:21.824958 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.823973    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18c8f8027c789794  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-08-05 17:12:21.814302612 +0000 UTC m=+0.347486904,LastTimestamp:2026-08-05 17:12:21.814302612 +0000 UTC m=+0.347486904,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Aug 05 17:12:21.835983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.835963    2562 manager.go:354] Recovery completed
Aug 05 17:12:21.836852 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.836823    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 05 17:12:21.837058 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.837031    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 05 17:12:21.842826 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.842811    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:21.844498 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.844484    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:21.844545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.844509    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:21.844545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.844520    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:21.845016 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845004    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 05 17:12:21.845016 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845015    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 05 17:12:21.845092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845020    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:21.845092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845028    2562 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:12:21.845736 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845720    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:21.845801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845747    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:21.845801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.845758    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:21.846158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.846147    2562 policy_none.go:49] "None policy: Start"
Aug 05 17:12:21.846621 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.846606    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 05 17:12:21.846664 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.846639    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 05 17:12:21.850416 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.850339    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18c8f8027e4555d3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-08-05 17:12:21.844497875 +0000 UTC m=+0.377682111,LastTimestamp:2026-08-05 17:12:21.844497875 +0000 UTC m=+0.377682111,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Aug 05 17:12:21.864034 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.863952    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18c8f8027e459612  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-08-05 17:12:21.844514322 +0000 UTC m=+0.377698558,LastTimestamp:2026-08-05 17:12:21.844514322 +0000 UTC m=+0.377698558,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Aug 05 17:12:21.866790 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.866773    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8cl6"
Aug 05 17:12:21.875478 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.875416    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18c8f8027e45bb49  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-08-05 17:12:21.844523849 +0000 UTC m=+0.377708085,LastTimestamp:2026-08-05 17:12:21.844523849 +0000 UTC m=+0.377708085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Aug 05 17:12:21.877134 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:21.877118    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8cl6"
Aug 05 17:12:21.922610 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:21.922581    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.000698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.000637    2562 policy_none.go:47] "Start"
Aug 05 17:12:22.025151 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.022867    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.025151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.024822    2562 manager.go:337] "Starting Device Plugin manager"
Aug 05 17:12:22.025151 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.024848    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 05 17:12:22.025151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.024859    2562 server.go:87] "Starting device plugin registration server"
Aug 05 17:12:22.025151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.025127    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 05 17:12:22.025151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.025139    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 05 17:12:22.025450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.025232    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 05 17:12:22.025450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.025290    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 05 17:12:22.025450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.025298    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 05 17:12:22.025891 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.025870    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 05 17:12:22.025975 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.025927    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.043517 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.043502    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-218.ec2.internal\" not found" node="ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.094684 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.094654    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 05 17:12:22.095891 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.095866    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 05 17:12:22.095982 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.095913    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 05 17:12:22.095982 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.095941    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 05 17:12:22.096047 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.095981    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 05 17:12:22.102234 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.102217    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:22.125523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.125506    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:22.126424 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.126409    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:22.126501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.126439    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:22.126501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.126452    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:22.126501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.126473    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.136624 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.136611    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.136667 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.136632    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-218.ec2.internal\": node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.149032 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.149013    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.196191 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.196016    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"]
Aug 05 17:12:22.196271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.196250    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:22.197014 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.197001    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:22.197067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.197028    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:22.197067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.197040    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:22.199085 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199074    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:22.199200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199187    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.199233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199216    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:22.199719 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199706    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:22.199719 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199712    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:22.199815 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199732    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:22.199815 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199744    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:22.199815 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199734    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:22.199897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.199838    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:22.201668 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.201654    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.201711 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.201682    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:22.202327 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.202294    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:22.202381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.202341    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:22.202381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.202358    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:22.224681 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.224659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fc06333c90d7f8d0e9adc3a7301a03c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"8fc06333c90d7f8d0e9adc3a7301a03c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.224772 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.224684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc06333c90d7f8d0e9adc3a7301a03c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"8fc06333c90d7f8d0e9adc3a7301a03c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.224772 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.224700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ea61056bac35de2c231d3be6b5b93be-config\") pod \"kube-apiserver-proxy-ip-10-0-131-218.ec2.internal\" (UID: \"0ea61056bac35de2c231d3be6b5b93be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.226978 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.226958    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-218.ec2.internal\" not found" node="ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.231270 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.231256    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-218.ec2.internal\" not found" node="ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.249477 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.249459    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.325178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.325117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fc06333c90d7f8d0e9adc3a7301a03c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"8fc06333c90d7f8d0e9adc3a7301a03c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.325178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.325143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc06333c90d7f8d0e9adc3a7301a03c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"8fc06333c90d7f8d0e9adc3a7301a03c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.325178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.325159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ea61056bac35de2c231d3be6b5b93be-config\") pod \"kube-apiserver-proxy-ip-10-0-131-218.ec2.internal\" (UID: \"0ea61056bac35de2c231d3be6b5b93be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.325401 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.325192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8fc06333c90d7f8d0e9adc3a7301a03c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"8fc06333c90d7f8d0e9adc3a7301a03c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.325401 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.325204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fc06333c90d7f8d0e9adc3a7301a03c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"8fc06333c90d7f8d0e9adc3a7301a03c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.325401 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.325197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ea61056bac35de2c231d3be6b5b93be-config\") pod \"kube-apiserver-proxy-ip-10-0-131-218.ec2.internal\" (UID: \"0ea61056bac35de2c231d3be6b5b93be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.350215 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.350198    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.450712 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.450681    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.529177 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.529144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.534166 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.533996    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.551719 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.551700    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.652256 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.652190    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.754324 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:22.754288    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Aug 05 17:12:22.761621 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.761595    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 05 17:12:22.761766 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.761738    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:12:22.822003 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.821983    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:22.823059 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.823032    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.823125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.823105    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 05 17:12:22.824089 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.824071    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:22.835196 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.835171    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:12:22.838167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.838155    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:12:22.840106 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.840093    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Aug 05 17:12:22.847654 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.847641    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:12:22.880450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.880411    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 17:07:22 +0000 UTC" deadline="2028-02-28 03:32:49.544696483 +0000 UTC"
Aug 05 17:12:22.880450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.880445    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13714h20m26.664255755s"
Aug 05 17:12:22.886840 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.886821    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8c8lf"
Aug 05 17:12:22.899664 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:22.899649    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8c8lf"
Aug 05 17:12:23.207793 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.207759    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:23.461540 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:23.461464    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea61056bac35de2c231d3be6b5b93be.slice/crio-775af785a6f19d2778f9538ec2a26cea3d45f2dc22697a76ff9239df4fa0f478 WatchSource:0}: Error finding container 775af785a6f19d2778f9538ec2a26cea3d45f2dc22697a76ff9239df4fa0f478: Status 404 returned error can't find the container with id 775af785a6f19d2778f9538ec2a26cea3d45f2dc22697a76ff9239df4fa0f478
Aug 05 17:12:23.468141 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.468121    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:12:23.474574 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:23.474552    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc06333c90d7f8d0e9adc3a7301a03c.slice/crio-773d18de32e1111d8c916cc673aef8511774f759845ed36074d7070d54fa7345 WatchSource:0}: Error finding container 773d18de32e1111d8c916cc673aef8511774f759845ed36074d7070d54fa7345: Status 404 returned error can't find the container with id 773d18de32e1111d8c916cc673aef8511774f759845ed36074d7070d54fa7345
Aug 05 17:12:23.638722 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.638697    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:23.806008 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.805939    2562 apiserver.go:52] "Watching apiserver"
Aug 05 17:12:23.814098 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.814080    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 05 17:12:23.816441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.816419    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-452n4","kube-system/konnectivity-agent-6grt6","kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal","openshift-cluster-node-tuning-operator/tuned-999q8","openshift-network-operator/iptables-alerter-5954k","openshift-ovn-kubernetes/ovnkube-node-mc8cp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd","openshift-image-registry/node-ca-69j8c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal","openshift-multus/multus-additional-cni-plugins-fw2rc","openshift-multus/multus-rvnmj","openshift-multus/network-metrics-daemon-pg7tl","openshift-network-diagnostics/network-check-target-fgk52"]
Aug 05 17:12:23.819430 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.819410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.821660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.821641    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.822405 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.822379    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 05 17:12:23.822501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.822425    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 05 17:12:23.822548 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.822498    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pzdzw\""
Aug 05 17:12:23.822710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.822685    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 05 17:12:23.823876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.823857    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.824132 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.824021    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:23.824284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.824258    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:23.824424 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.824364    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ktxc7\""
Aug 05 17:12:23.826238 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.826216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:23.826343 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.826325    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:23.826622 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.826603    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 05 17:12:23.827218 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.827200    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:23.827623 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.827606    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5fh8h\""
Aug 05 17:12:23.828634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.828619    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.830924 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.830901    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.831582 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.831566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 05 17:12:23.831684 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.831570    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 05 17:12:23.831811 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.831796    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 05 17:12:23.831893 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.831878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tk49d\""
Aug 05 17:12:23.833144 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833126    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:23.833239 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-registration-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.833239 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833204    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysctl-d\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.833239 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:23.833203    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:23.833239 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-systemd\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.833486 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-host\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.833486 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caec4e7e-0f68-4de9-92b2-057375c14245-iptables-alerter-script\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.833676 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnsl8\" (UniqueName: \"kubernetes.io/projected/caec4e7e-0f68-4de9-92b2-057375c14245-kube-api-access-cnsl8\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.833737 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-device-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.833797 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-sys-fs\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.833797 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833771    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysconfig\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.834011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-kubernetes\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.834011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-run\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.834011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-var-lib-kubelet\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.834011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833945    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf7jd\" (UniqueName: \"kubernetes.io/projected/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-kube-api-access-nf7jd\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.834011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.833987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.834275 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.834026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-socket-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.834275 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.834078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.836420 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-modprobe-d\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.836511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysctl-conf\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.836568 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-sys\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.836624 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-tmp\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.836667 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836630    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caec4e7e-0f68-4de9-92b2-057375c14245-host-slash\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.836983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836967    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp49m\" (UniqueName: \"kubernetes.io/projected/b8b358b4-85b3-4596-9d60-7d2d0276fb72-kube-api-access-kp49m\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.837023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.836992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-lib-modules\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.837023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.837009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-tuned\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.837120 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.837067    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 05 17:12:23.837193 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.837178    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rm5jf\""
Aug 05 17:12:23.837546 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.837531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:23.837605 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:23.837586    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:23.838412 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.838394    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 05 17:12:23.838703 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.838688    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 05 17:12:23.838759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.838727    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 05 17:12:23.838812 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.838689    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 05 17:12:23.838866 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.838806    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 05 17:12:23.838866 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.838856    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tbcff\""
Aug 05 17:12:23.839935 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.839906    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:23.842509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.842486    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.843558 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.843533    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dz5h7\""
Aug 05 17:12:23.843558 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.843533    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 05 17:12:23.843698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.843536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 05 17:12:23.844924 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.844905    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 05 17:12:23.845002 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.844908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:23.845043 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:23.845001    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:23.847474 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.847457    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 05 17:12:23.847569 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.847481    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 05 17:12:23.847635 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.847582    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 05 17:12:23.848414 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.848395    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 05 17:12:23.848498 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.848464    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 05 17:12:23.848498 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.848494    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pxqls\""
Aug 05 17:12:23.902739 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.902684    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:07:23 +0000 UTC" deadline="2028-01-06 19:14:41.755293047 +0000 UTC"
Aug 05 17:12:23.902739 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.902731    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12458h2m17.852564929s"
Aug 05 17:12:23.924176 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.924153    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 05 17:12:23.937295 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-device-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.937426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-var-lib-kubelet\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.937426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.937426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-log-socket\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.937426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-host\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:23.937426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b6d9b10-5dbb-4b5e-9900-226cde1d2d48-konnectivity-ca\") pod \"konnectivity-agent-6grt6\" (UID: \"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48\") " pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:23.937426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-tmp\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937497    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caec4e7e-0f68-4de9-92b2-057375c14245-host-slash\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-device-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937551    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caec4e7e-0f68-4de9-92b2-057375c14245-host-slash\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-netns\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-var-lib-kubelet\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-daemon-config\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp49m\" (UniqueName: \"kubernetes.io/projected/b8b358b4-85b3-4596-9d60-7d2d0276fb72-kube-api-access-kp49m\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.937669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4zft\" (UniqueName: \"kubernetes.io/projected/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-kube-api-access-q4zft\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-cni-bin\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-registration-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b6d9b10-5dbb-4b5e-9900-226cde1d2d48-agent-certs\") pod \"konnectivity-agent-6grt6\" (UID: \"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48\") " pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnsl8\" (UniqueName: \"kubernetes.io/projected/caec4e7e-0f68-4de9-92b2-057375c14245-kube-api-access-cnsl8\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-socket-dir-parent\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-registration-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-conf-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-etc-kubernetes\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937929    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.937983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysctl-conf\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-cnibin\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.938092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938087    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-cni-multus\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-kubelet-config\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938136    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clbkk\" (UniqueName: \"kubernetes.io/projected/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-kube-api-access-clbkk\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-lib-modules\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysctl-conf\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-slash\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovnkube-script-lib\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caec4e7e-0f68-4de9-92b2-057375c14245-iptables-alerter-script\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-systemd-units\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-node-log\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-cni-netd\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovn-node-metrics-cert\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-lib-modules\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-dbus\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-sys-fs\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysconfig\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-sys-fs\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.938738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-run\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-hostroot\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938641    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysconfig\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-run\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938796    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-cni-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938801    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-tuned\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cnibin\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caec4e7e-0f68-4de9-92b2-057375c14245-iptables-alerter-script\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.938955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-multus-certs\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.939466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939031    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-kubelet\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-run-netns\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-system-cni-dir\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-os-release\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-cni-bin\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-etc-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939255    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwn6q\" (UniqueName: \"kubernetes.io/projected/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-kube-api-access-nwn6q\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939292    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-kubernetes\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf7jd\" (UniqueName: \"kubernetes.io/projected/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-kube-api-access-nf7jd\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-kubernetes\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-os-release\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxc7\" (UniqueName: \"kubernetes.io/projected/8d27b52c-4637-4d6c-b323-4da630c5a6b0-kube-api-access-twxc7\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939550    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-system-cni-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s87kl\" (UniqueName: \"kubernetes.io/projected/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-kube-api-access-s87kl\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939620    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-socket-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939650    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-modprobe-d\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-sys\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-cni-binary-copy\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-k8s-cni-cncf-io\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-modprobe-d\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b8b358b4-85b3-4596-9d60-7d2d0276fb72-socket-dir\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-systemd\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-sys\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-ovn\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-serviceca\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-var-lib-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovnkube-config\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-env-overrides\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysctl-d\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.939985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-systemd\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.940012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-host\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.940671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.940040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-kubelet\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:23.941174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.940043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-systemd\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.941174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.940067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-sysctl-d\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.941174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.940148    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-host\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.941594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.941577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-tmp\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.941645 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.941625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-etc-tuned\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.954209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.954179    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf7jd\" (UniqueName: \"kubernetes.io/projected/a2b01705-9134-4ebf-8ba6-83c4d07ad1cc-kube-api-access-nf7jd\") pod \"tuned-999q8\" (UID: \"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc\") " pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:23.954300 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.954278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp49m\" (UniqueName: \"kubernetes.io/projected/b8b358b4-85b3-4596-9d60-7d2d0276fb72-kube-api-access-kp49m\") pod \"aws-ebs-csi-driver-node-t4pwd\" (UID: \"b8b358b4-85b3-4596-9d60-7d2d0276fb72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:23.954381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:23.954362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnsl8\" (UniqueName: \"kubernetes.io/projected/caec4e7e-0f68-4de9-92b2-057375c14245-kube-api-access-cnsl8\") pod \"iptables-alerter-5954k\" (UID: \"caec4e7e-0f68-4de9-92b2-057375c14245\") " pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:24.040621 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-socket-dir-parent\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-conf-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-etc-kubernetes\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-socket-dir-parent\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-cnibin\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-cni-multus\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.040775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-etc-kubernetes\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040801    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-kubelet-config\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-conf-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-cnibin\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clbkk\" (UniqueName: \"kubernetes.io/projected/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-kube-api-access-clbkk\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-slash\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-cni-multus\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-slash\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovnkube-script-lib\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-systemd-units\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.040987    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-node-log\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041010    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-cni-netd\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-cni-netd\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-systemd-units\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041132    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-node-log\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-kubelet-config\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovn-node-metrics-cert\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-dbus\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-hostroot\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041374    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.041463    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovnkube-script-lib\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.041540    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:24.541518203 +0000 UTC m=+3.074702432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-cni-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cnibin\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-hostroot\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.041989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-dbus\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-multus-certs\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-kubelet\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-cni-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041777    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-run-netns\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-run-netns\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-system-cni-dir\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-os-release\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-kubelet\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-cni-bin\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041930    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-system-cni-dir\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.041938    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-etc-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-cni-bin\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-multus-certs\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.042029    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:12:24.542010162 +0000 UTC m=+3.075194387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-os-release\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.042897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.041931    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-etc-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-run-ovn-kubernetes\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042072    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cnibin\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwn6q\" (UniqueName: \"kubernetes.io/projected/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-kube-api-access-nwn6q\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-os-release\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxc7\" (UniqueName: \"kubernetes.io/projected/8d27b52c-4637-4d6c-b323-4da630c5a6b0-kube-api-access-twxc7\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-system-cni-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s87kl\" (UniqueName: \"kubernetes.io/projected/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-kube-api-access-s87kl\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-cni-binary-copy\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-k8s-cni-cncf-io\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042381    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-systemd\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042416    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d27b52c-4637-4d6c-b323-4da630c5a6b0-os-release\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-ovn\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.043802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8d27b52c-4637-4d6c-b323-4da630c5a6b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042484    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-k8s-cni-cncf-io\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-serviceca\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-systemd\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-var-lib-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-ovn\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovnkube-config\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-system-cni-dir\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042598    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-env-overrides\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-kubelet\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-log-socket\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-host\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b6d9b10-5dbb-4b5e-9900-226cde1d2d48-konnectivity-ca\") pod \"konnectivity-agent-6grt6\" (UID: \"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48\") " pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042827    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-run-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-netns\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042897    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-daemon-config\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4zft\" (UniqueName: \"kubernetes.io/projected/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-kube-api-access-q4zft\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.044480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-serviceca\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-cni-bin\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b6d9b10-5dbb-4b5e-9900-226cde1d2d48-agent-certs\") pod \"konnectivity-agent-6grt6\" (UID: \"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48\") " pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.042999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-env-overrides\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043004    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-log-socket\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-cni-binary-copy\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-var-lib-openvswitch\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-var-lib-kubelet\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043177    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-host-cni-bin\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043183    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-host\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-host-run-netns\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b6d9b10-5dbb-4b5e-9900-226cde1d2d48-konnectivity-ca\") pod \"konnectivity-agent-6grt6\" (UID: \"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48\") " pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovnkube-config\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-multus-daemon-config\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.045202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.043963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-ovn-node-metrics-cert\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.045779 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.045757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b6d9b10-5dbb-4b5e-9900-226cde1d2d48-agent-certs\") pod \"konnectivity-agent-6grt6\" (UID: \"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48\") " pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:24.053670 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.053648    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:24.054394 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.053673    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:24.054394 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.053688    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:24.054394 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.053741    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:24.553728759 +0000 UTC m=+3.086913015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:24.055201 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.055173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clbkk\" (UniqueName: \"kubernetes.io/projected/f58e0111-2bdd-4b12-8da2-d65cbf3de0b5-kube-api-access-clbkk\") pod \"node-ca-69j8c\" (UID: \"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5\") " pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.055768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.055745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwn6q\" (UniqueName: \"kubernetes.io/projected/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-kube-api-access-nwn6q\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:24.056614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.056547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4zft\" (UniqueName: \"kubernetes.io/projected/b1b9c98a-0040-4e90-949b-b16d2b3ffea8-kube-api-access-q4zft\") pod \"multus-rvnmj\" (UID: \"b1b9c98a-0040-4e90-949b-b16d2b3ffea8\") " pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.059465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.059446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxc7\" (UniqueName: \"kubernetes.io/projected/8d27b52c-4637-4d6c-b323-4da630c5a6b0-kube-api-access-twxc7\") pod \"multus-additional-cni-plugins-fw2rc\" (UID: \"8d27b52c-4637-4d6c-b323-4da630c5a6b0\") " pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.059828 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.059803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s87kl\" (UniqueName: \"kubernetes.io/projected/ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb-kube-api-access-s87kl\") pod \"ovnkube-node-mc8cp\" (UID: \"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb\") " pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.103640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.103580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"8fc06333c90d7f8d0e9adc3a7301a03c","Type":"ContainerStarted","Data":"773d18de32e1111d8c916cc673aef8511774f759845ed36074d7070d54fa7345"}
Aug 05 17:12:24.104848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.104823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal" event={"ID":"0ea61056bac35de2c231d3be6b5b93be","Type":"ContainerStarted","Data":"775af785a6f19d2778f9538ec2a26cea3d45f2dc22697a76ff9239df4fa0f478"}
Aug 05 17:12:24.131436 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.131413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd"
Aug 05 17:12:24.139142 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.139122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-999q8"
Aug 05 17:12:24.143495 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.143468    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b358b4_85b3_4596_9d60_7d2d0276fb72.slice/crio-b05a1d750849a6d382f38e85f11cecc9148df6c3b8fd1ef45c6e4ade698632bd WatchSource:0}: Error finding container b05a1d750849a6d382f38e85f11cecc9148df6c3b8fd1ef45c6e4ade698632bd: Status 404 returned error can't find the container with id b05a1d750849a6d382f38e85f11cecc9148df6c3b8fd1ef45c6e4ade698632bd
Aug 05 17:12:24.146671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.146649    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5954k"
Aug 05 17:12:24.149173 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.149147    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b01705_9134_4ebf_8ba6_83c4d07ad1cc.slice/crio-bb73b37cd28033d164b9a894fadc2b89efbfec82f4997e2d03657319de10fb3e WatchSource:0}: Error finding container bb73b37cd28033d164b9a894fadc2b89efbfec82f4997e2d03657319de10fb3e: Status 404 returned error can't find the container with id bb73b37cd28033d164b9a894fadc2b89efbfec82f4997e2d03657319de10fb3e
Aug 05 17:12:24.152858 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.152836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-69j8c"
Aug 05 17:12:24.156734 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.156710    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaec4e7e_0f68_4de9_92b2_057375c14245.slice/crio-a139bd711f7ad38f6160968e4a4b72c5f71eb8d5c6cbaf6b7608838877fcc930 WatchSource:0}: Error finding container a139bd711f7ad38f6160968e4a4b72c5f71eb8d5c6cbaf6b7608838877fcc930: Status 404 returned error can't find the container with id a139bd711f7ad38f6160968e4a4b72c5f71eb8d5c6cbaf6b7608838877fcc930
Aug 05 17:12:24.157620 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.157596    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fw2rc"
Aug 05 17:12:24.162292 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.162260    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58e0111_2bdd_4b12_8da2_d65cbf3de0b5.slice/crio-b974194aaa54026092122e8d730e56e89733d34390fd53a06c54b5e486d1ca86 WatchSource:0}: Error finding container b974194aaa54026092122e8d730e56e89733d34390fd53a06c54b5e486d1ca86: Status 404 returned error can't find the container with id b974194aaa54026092122e8d730e56e89733d34390fd53a06c54b5e486d1ca86
Aug 05 17:12:24.163564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.163546    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rvnmj"
Aug 05 17:12:24.167984 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.167961    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d27b52c_4637_4d6c_b323_4da630c5a6b0.slice/crio-4c707ea50db8dd363a86cfdf12f18f470503bafcdeb4a340fef69e24f0da160f WatchSource:0}: Error finding container 4c707ea50db8dd363a86cfdf12f18f470503bafcdeb4a340fef69e24f0da160f: Status 404 returned error can't find the container with id 4c707ea50db8dd363a86cfdf12f18f470503bafcdeb4a340fef69e24f0da160f
Aug 05 17:12:24.169604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.169577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:24.175617 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.175593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:24.177573 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.177519    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b9c98a_0040_4e90_949b_b16d2b3ffea8.slice/crio-b93ac079259b3b74a4a0889c6ef49b247b22c6f8fc7138af5aafa81d918929f2 WatchSource:0}: Error finding container b93ac079259b3b74a4a0889c6ef49b247b22c6f8fc7138af5aafa81d918929f2: Status 404 returned error can't find the container with id b93ac079259b3b74a4a0889c6ef49b247b22c6f8fc7138af5aafa81d918929f2
Aug 05 17:12:24.187058 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:24.187028    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6ecaeb_46b2_4555_92f1_26d2f7dff5eb.slice/crio-40c866fa9e84b99fdbc4a77e063fe237421d967129616f11d9345c8769025928 WatchSource:0}: Error finding container 40c866fa9e84b99fdbc4a77e063fe237421d967129616f11d9345c8769025928: Status 404 returned error can't find the container with id 40c866fa9e84b99fdbc4a77e063fe237421d967129616f11d9345c8769025928
Aug 05 17:12:24.547095 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.546890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:24.547095 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.546949    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:24.547095 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.547015    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:24.547095 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.547052    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:24.547095 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.547083    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:25.547063907 +0000 UTC m=+4.080248153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:24.547095 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.547101    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:12:25.547092792 +0000 UTC m=+4.080277224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:24.648082 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.648016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:24.648271 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.648253    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:24.648364 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.648280    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:24.648364 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.648294    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:24.648479 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:24.648376    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:25.648355764 +0000 UTC m=+4.181539992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:24.903674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.903546    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:07:23 +0000 UTC" deadline="2028-05-12 14:22:15.89661941 +0000 UTC"
Aug 05 17:12:24.903674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:24.903604    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15501h9m50.99301913s"
Aug 05 17:12:25.096698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.096654    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:25.096896 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.096799    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:25.108920 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.108877    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"40c866fa9e84b99fdbc4a77e063fe237421d967129616f11d9345c8769025928"}
Aug 05 17:12:25.110580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.110550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6grt6" event={"ID":"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48","Type":"ContainerStarted","Data":"dcbfd94e08efc2be61d92a22e1eaabe0592c80e2a01d17417c92683597666eaf"}
Aug 05 17:12:25.112542 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.112472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rvnmj" event={"ID":"b1b9c98a-0040-4e90-949b-b16d2b3ffea8","Type":"ContainerStarted","Data":"b93ac079259b3b74a4a0889c6ef49b247b22c6f8fc7138af5aafa81d918929f2"}
Aug 05 17:12:25.114868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.114841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5954k" event={"ID":"caec4e7e-0f68-4de9-92b2-057375c14245","Type":"ContainerStarted","Data":"a139bd711f7ad38f6160968e4a4b72c5f71eb8d5c6cbaf6b7608838877fcc930"}
Aug 05 17:12:25.117778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.117752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd" event={"ID":"b8b358b4-85b3-4596-9d60-7d2d0276fb72","Type":"ContainerStarted","Data":"b05a1d750849a6d382f38e85f11cecc9148df6c3b8fd1ef45c6e4ade698632bd"}
Aug 05 17:12:25.119361 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.119316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerStarted","Data":"4c707ea50db8dd363a86cfdf12f18f470503bafcdeb4a340fef69e24f0da160f"}
Aug 05 17:12:25.121467 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.121442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-69j8c" event={"ID":"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5","Type":"ContainerStarted","Data":"b974194aaa54026092122e8d730e56e89733d34390fd53a06c54b5e486d1ca86"}
Aug 05 17:12:25.123808 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.123779    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-999q8" event={"ID":"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc","Type":"ContainerStarted","Data":"bb73b37cd28033d164b9a894fadc2b89efbfec82f4997e2d03657319de10fb3e"}
Aug 05 17:12:25.557968 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.557926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:25.558145 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.557982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:25.558145 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.558120    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:25.558279 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.558186    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:12:27.558165378 +0000 UTC m=+6.091349611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:25.558763 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.558634    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:25.558763 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.558689    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:27.558673622 +0000 UTC m=+6.091857851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:25.658685 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:25.658616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:25.658899 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.658815    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:25.658899 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.658842    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:25.658899 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.658857    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:25.659070 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:25.658922    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:27.658903211 +0000 UTC m=+6.192087452 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:26.098160 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:26.097529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:26.098160 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:26.097695    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:26.099169 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:26.098967    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:26.099169 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:26.099084    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:27.096672 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:27.096634    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:27.096872 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.096802    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:27.579801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:27.579764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:27.580203 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:27.579817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:27.580203 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.579951    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:27.580203 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.580011    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:12:31.579992852 +0000 UTC m=+10.113177079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:27.580443 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.580424    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:27.580516 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.580497    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:31.580470827 +0000 UTC m=+10.113655077 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:27.680604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:27.680564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:27.680803 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.680784    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:27.680890 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.680816    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:27.680890 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.680830    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:27.681004 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:27.680887    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:31.680868583 +0000 UTC m=+10.214052811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:28.097494 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:28.097461    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:28.097675 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:28.097624    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:28.100205 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:28.100048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:28.100205 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:28.100158    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:29.096525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:29.096486    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:29.096956 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:29.096627    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:30.096929 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:30.096892    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:30.097473 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:30.097052    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:30.097558 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:30.097500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:30.097613 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:30.097589    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:31.096955 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:31.096919    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:31.097444 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.097068    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:31.617971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:31.617233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:31.617971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:31.617277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:31.617971 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.617451    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:31.617971 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.617516    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:12:39.61749779 +0000 UTC m=+18.150682037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:31.617971 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.617888    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:31.617971 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.617937    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:39.617922458 +0000 UTC m=+18.151106684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:31.717865 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:31.717825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:31.718050 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.718032    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:31.718116 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.718062    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:31.718116 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.718077    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:31.718216 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:31.718143    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:39.718124326 +0000 UTC m=+18.251308567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:32.096853 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:32.096785    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:32.097014 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:32.096943    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:32.097453 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:32.097392    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:32.097530 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:32.097494    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:33.096274 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:33.096241    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:33.096467 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:33.096385    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:34.096355 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:34.096326    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:34.096778 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:34.096456    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:34.096778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:34.096550    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:34.096778 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:34.096659    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:35.096616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:35.096580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:35.097076 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:35.096708    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:36.096915 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:36.096879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:36.097407 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:36.096887    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:36.097407 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:36.097036    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:36.097407 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:36.097075    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:37.096131 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:37.096096    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:37.096295 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:37.096208    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:38.096651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:38.096610    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:38.096651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:38.096642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:38.097151 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:38.096760    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:38.097151 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:38.096941    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:39.096582 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:39.096539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:39.096793 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.096679    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:39.678413 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:39.678325    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:39.678413 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:39.678389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:39.678670 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.678478    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:39.678670 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.678485    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:39.678670 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.678548    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:12:55.678533494 +0000 UTC m=+34.211717716 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:39.678670 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.678563    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:55.678556878 +0000 UTC m=+34.211741101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:39.779605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:39.779563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:39.779798 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.779780    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:39.779869 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.779807    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:39.779869 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.779821    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:39.779955 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:39.779888    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:55.779867638 +0000 UTC m=+34.313051861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:40.096328 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:40.096282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:40.096507 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:40.096434    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:40.096507 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:40.096495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:40.096642 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:40.096622    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:41.097134 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:41.097092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:41.097602 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:41.097231    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:42.099333 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:42.099293    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:42.099732 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:42.099403    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:42.099732 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:42.099441    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:42.099732 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:42.099500    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:43.096613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:43.096560    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:43.096753 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:43.096710    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:44.097840 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.096832    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:44.097840 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.097092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:44.097840 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:44.097463    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:44.097840 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:44.097550    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:44.164094 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.163637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal" event={"ID":"0ea61056bac35de2c231d3be6b5b93be","Type":"ContainerStarted","Data":"739f3449b8ae0b87baccc6308ba8d74861384f7dfd562902c2a25e825fb8561a"}
Aug 05 17:12:44.170839 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.170232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-999q8" event={"ID":"a2b01705-9134-4ebf-8ba6-83c4d07ad1cc","Type":"ContainerStarted","Data":"0ed57e8e5186fd7883b58b3a46242d67abedf5b5946767ea0c4b193699c83376"}
Aug 05 17:12:44.176652 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.176630    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:12:44.177188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.176981    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb" containerID="74951f5cdc1ae5223ea323dc96c1dff005d9e272514d4e7905dec77abc6f3973" exitCode=1
Aug 05 17:12:44.177188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.177006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"93a002efc33fb71e232c01f0ee4d5219be86c3a4cae32e8ddbdec8b6bcffd13d"}
Aug 05 17:12:44.177188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.177084    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"fb202f31306d7295b7512a1762ea2799d68fc0d2629aa0c8d2c528897d2e9700"}
Aug 05 17:12:44.177188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.177135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"df6e90f69969a7245a00f66a3c499f247c2ade231708cd6114a0625625401e62"}
Aug 05 17:12:44.177188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.177186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"0c8095d9e22489f2f3b45ec33c3ec97bd2daed47e2c8d8c237bca66388999b55"}
Aug 05 17:12:44.177452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.177235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerDied","Data":"74951f5cdc1ae5223ea323dc96c1dff005d9e272514d4e7905dec77abc6f3973"}
Aug 05 17:12:44.177452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.177284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"7ecdf1b465f3609a78f3181ae0aa23fcec6773562b418764873d6b9bb85d83c0"}
Aug 05 17:12:44.178896 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.178794    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6grt6" event={"ID":"0b6d9b10-5dbb-4b5e-9900-226cde1d2d48","Type":"ContainerStarted","Data":"e23666de7723638fa46f6290509f0bbe3b54603b2d7ab604319866589fb82b76"}
Aug 05 17:12:44.180961 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.180692    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rvnmj" event={"ID":"b1b9c98a-0040-4e90-949b-b16d2b3ffea8","Type":"ContainerStarted","Data":"6888472f2eec49fe713a07b95cb078fa83efd97fee0e4bb8d68a2604d01ecc97"}
Aug 05 17:12:44.192468 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.192428    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal" podStartSLOduration=21.192414659 podStartE2EDuration="21.192414659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:44.192157963 +0000 UTC m=+22.725342228" watchObservedRunningTime="2026-08-05 17:12:44.192414659 +0000 UTC m=+22.725598903"
Aug 05 17:12:44.217819 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.217778    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rvnmj" podStartSLOduration=2.572117811 podStartE2EDuration="22.217762351s" totalImagesPullingTime="19.64564454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:44.217307596 +0000 UTC m=+22.750491842" watchObservedRunningTime="2026-08-05 17:12:44.217762351 +0000 UTC m=+22.750946597"
Aug 05 17:12:44.248332 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.248183    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6grt6" podStartSLOduration=1.959789861 podStartE2EDuration="21.248164929s" totalImagesPullingTime="19.288375068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:44.247947493 +0000 UTC m=+22.781131739" watchObservedRunningTime="2026-08-05 17:12:44.248164929 +0000 UTC m=+22.781349173"
Aug 05 17:12:44.263674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:44.263633    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-999q8" podStartSLOduration=2.944283912 podStartE2EDuration="22.263618164s" totalImagesPullingTime="19.319334252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:44.263535745 +0000 UTC m=+22.796719990" watchObservedRunningTime="2026-08-05 17:12:44.263618164 +0000 UTC m=+22.796802406"
Aug 05 17:12:45.097191 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.097157    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:45.097386 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:45.097288    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:45.185848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.185761    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d27b52c-4637-4d6c-b323-4da630c5a6b0" containerID="9a3bdb890f6cb3b7ea6166c96b5bf3da2f32eef5273752de97aad80dd8ec096e" exitCode=0
Aug 05 17:12:45.185848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.185814    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerDied","Data":"9a3bdb890f6cb3b7ea6166c96b5bf3da2f32eef5273752de97aad80dd8ec096e"}
Aug 05 17:12:45.187634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.187608    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-69j8c" event={"ID":"f58e0111-2bdd-4b12-8da2-d65cbf3de0b5","Type":"ContainerStarted","Data":"e17e21fc810ddfd8077e726e888941f5787390bf1acf38b6b6949ba0ef3ab500"}
Aug 05 17:12:45.189136 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.189115    2562 generic.go:356] "Generic (PLEG): container finished" podID="8fc06333c90d7f8d0e9adc3a7301a03c" containerID="ada9ea4057efeb22bd6d518328def03d06978f3dd0289790c00a85264d0bf248" exitCode=0
Aug 05 17:12:45.189296 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.189197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"8fc06333c90d7f8d0e9adc3a7301a03c","Type":"ContainerDied","Data":"ada9ea4057efeb22bd6d518328def03d06978f3dd0289790c00a85264d0bf248"}
Aug 05 17:12:45.191150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.190811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5954k" event={"ID":"caec4e7e-0f68-4de9-92b2-057375c14245","Type":"ContainerStarted","Data":"134a0cea66215d663f6ea6b0a1eb2cf738237749aaeb066db9f1532e012371aa"}
Aug 05 17:12:45.192374 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.192309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd" event={"ID":"b8b358b4-85b3-4596-9d60-7d2d0276fb72","Type":"ContainerStarted","Data":"b4bc9e35d4ab6a86be6a7dcf592c82a4e3b2063e28dfe4b6eceac9cb6f6ef54d"}
Aug 05 17:12:45.237087 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.236855    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5954k" podStartSLOduration=3.952268155 podStartE2EDuration="23.236834405s" totalImagesPullingTime="19.28456625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:45.236676999 +0000 UTC m=+23.769861264" watchObservedRunningTime="2026-08-05 17:12:45.236834405 +0000 UTC m=+23.770018651"
Aug 05 17:12:45.252473 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.252409    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-69j8c" podStartSLOduration=12.805343638 podStartE2EDuration="23.25239348s" totalImagesPullingTime="10.447049842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:45.251903079 +0000 UTC m=+23.785087325" watchObservedRunningTime="2026-08-05 17:12:45.25239348 +0000 UTC m=+23.785577725"
Aug 05 17:12:45.819504 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:45.819478    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 05 17:12:46.037828 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.037676    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T17:12:45.819499688Z","UUID":"a8a21463-dc0d-4df6-9a3e-3e488e21b8cb","Handler":null,"Name":"","Endpoint":""}
Aug 05 17:12:46.039389 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.039367    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 05 17:12:46.039496 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.039407    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 05 17:12:46.097032 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.097007    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:46.097198 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:46.097141    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:46.097198 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.097179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:46.097328 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:46.097253    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:46.196569 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.196500    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"8fc06333c90d7f8d0e9adc3a7301a03c","Type":"ContainerStarted","Data":"c0d08c98e27901d125d36fc6e0a793c61a95b98b8b844e87208dc8ec9d841eac"}
Aug 05 17:12:46.198865 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.198829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd" event={"ID":"b8b358b4-85b3-4596-9d60-7d2d0276fb72","Type":"ContainerStarted","Data":"f7f59c2c8da21eda38b40c683563c31dc94200caf59df9eaf51b738cf03df6aa"}
Aug 05 17:12:46.214695 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:46.214655    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" podStartSLOduration=23.214644189 podStartE2EDuration="23.214644189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:46.214396094 +0000 UTC m=+24.747580338" watchObservedRunningTime="2026-08-05 17:12:46.214644189 +0000 UTC m=+24.747828434"
Aug 05 17:12:47.096348 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:47.096305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:47.096489 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:47.096415    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:47.202907 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:47.202820    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd" event={"ID":"b8b358b4-85b3-4596-9d60-7d2d0276fb72","Type":"ContainerStarted","Data":"c382af6eb155307f5a83fd99ff05a2106d371b8bd9099c8f76809299042fac26"}
Aug 05 17:12:47.206583 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:47.206542    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:12:47.207028 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:47.207001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"4fc3bac379e42f3db7729a6096b7017db3c1bb246679a7cf1e29a1ff3aa0bf85"}
Aug 05 17:12:47.222760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:47.222713    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4pwd" podStartSLOduration=3.289625381 podStartE2EDuration="25.22269547s" totalImagesPullingTime="21.933070089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:47.222295902 +0000 UTC m=+25.755480138" watchObservedRunningTime="2026-08-05 17:12:47.22269547 +0000 UTC m=+25.755879717"
Aug 05 17:12:48.096966 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:48.096933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:48.097137 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:48.097050    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:48.097137 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:48.097102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:48.097229 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:48.097188    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:48.918253 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:48.918008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:48.918780 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:48.918760    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:49.096163 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.096134    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:49.096446 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:49.096277    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:49.216823 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.216792    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:12:49.217280 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.217247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"d4c930dff34b0c27b2999bf0c342cbc5c380b072f85f5bea3f80d0121c2ba427"}
Aug 05 17:12:49.217670 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.217644    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:49.217796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.217756    2562 scope.go:117] "RemoveContainer" containerID="74951f5cdc1ae5223ea323dc96c1dff005d9e272514d4e7905dec77abc6f3973"
Aug 05 17:12:49.217796 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.217772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:49.218528 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.218465    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6grt6"
Aug 05 17:12:49.235299 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:49.235278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:50.096564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.096537    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:50.097084 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.096584    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:50.097084 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:50.096657    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:50.097084 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:50.096716    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:50.223916 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.223895    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:12:50.224341 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.224286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" event={"ID":"ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb","Type":"ContainerStarted","Data":"015a887c7cb93a8c2c3b28da1f00706bc24f8753958aaa0e33cbe3ac8d65c676"}
Aug 05 17:12:50.224606 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.224583    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:50.224801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.224777    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:50.242133 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.242109    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:12:50.255972 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.255933    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp" podStartSLOduration=7.928125067 podStartE2EDuration="27.255920333s" totalImagesPullingTime="19.327795266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:50.255666928 +0000 UTC m=+28.788851172" watchObservedRunningTime="2026-08-05 17:12:50.255920333 +0000 UTC m=+28.789104577"
Aug 05 17:12:50.379749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.379726    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fgk52"]
Aug 05 17:12:50.379853 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.379825    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:50.379930 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:50.379905    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:50.382509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.382489    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-452n4"]
Aug 05 17:12:50.382598 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.382571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:50.382666 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:50.382637    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:50.396839 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.396819    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pg7tl"]
Aug 05 17:12:50.396928 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:50.396905    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:50.396990 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:50.396975    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:51.228470 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:51.228437    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d27b52c-4637-4d6c-b323-4da630c5a6b0" containerID="57030decf8e28c041f84c1e859b276c249c77bf3ecfa97fb906aed38cb964366" exitCode=0
Aug 05 17:12:51.228965 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:51.228525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerDied","Data":"57030decf8e28c041f84c1e859b276c249c77bf3ecfa97fb906aed38cb964366"}
Aug 05 17:12:52.099351 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:52.099309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:52.099351 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:52.099334    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:52.099534 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:52.099334    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:52.099534 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:52.099406    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:52.099534 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:52.099497    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:52.099634 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:52.099558    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:53.234918 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:53.234839    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d27b52c-4637-4d6c-b323-4da630c5a6b0" containerID="d11c2cb9cbe3b0e5628aa6f3c0e105e27b6c120d8280cec415e4216fd27a9ac0" exitCode=0
Aug 05 17:12:53.234918 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:53.234883    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerDied","Data":"d11c2cb9cbe3b0e5628aa6f3c0e105e27b6c120d8280cec415e4216fd27a9ac0"}
Aug 05 17:12:54.097179 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:54.097146    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:54.097179 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:54.097181    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:54.097430 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:54.097146    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:54.097430 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:54.097272    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:54.097430 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:54.097390    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:54.097560 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:54.097479    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:55.241958 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.241923    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d27b52c-4637-4d6c-b323-4da630c5a6b0" containerID="6263eadcf1200fd9fdb2d96b9e88447f20b18736f5e91ef946a2831f353472dc" exitCode=0
Aug 05 17:12:55.242507 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.241980    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerDied","Data":"6263eadcf1200fd9fdb2d96b9e88447f20b18736f5e91ef946a2831f353472dc"}
Aug 05 17:12:55.693966 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.693934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:55.693966 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.693969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:55.694157 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.694072    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:55.694157 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.694073    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:55.694157 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.694122    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs podName:1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa nodeName:}" failed. No retries permitted until 2026-08-05 17:13:27.694108243 +0000 UTC m=+66.227292466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs") pod "network-metrics-daemon-pg7tl" (UID: "1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:55.694157 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.694134    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret podName:bc5be61d-7ca6-4860-ac6d-92cf4d56ac51 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:27.69412875 +0000 UTC m=+66.227312972 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret") pod "global-pull-secret-syncer-452n4" (UID: "bc5be61d-7ca6-4860-ac6d-92cf4d56ac51") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:55.795289 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.795203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:55.795465 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.795377    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:55.795465 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.795396    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:55.795465 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.795407    2562 projected.go:196] Error preparing data for projected volume kube-api-access-vtn6t for pod openshift-network-diagnostics/network-check-target-fgk52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:55.795465 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:55.795458    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t podName:a16139f2-7cad-4290-9562-7b0d79df30f8 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:27.795445844 +0000 UTC m=+66.328630067 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vtn6t" (UniqueName: "kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t") pod "network-check-target-fgk52" (UID: "a16139f2-7cad-4290-9562-7b0d79df30f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:55.818545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.818520    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-2czsm"]
Aug 05 17:12:55.834185 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.834148    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:55.837615 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.837393    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cfcdk\""
Aug 05 17:12:55.837732 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.837394    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 05 17:12:55.837732 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.837480    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 05 17:12:55.996937 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.996900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-hosts-file\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:55.997114 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.996971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-tmp-dir\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:55.997114 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:55.997041    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vx2t\" (UniqueName: \"kubernetes.io/projected/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-kube-api-access-9vx2t\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.097574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.097491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-hosts-file\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.097574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.097567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-tmp-dir\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.097769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.097594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vx2t\" (UniqueName: \"kubernetes.io/projected/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-kube-api-access-9vx2t\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.097769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.097622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-hosts-file\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.099778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.099756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:56.099876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.099863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:56.099990 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:56.099864    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-452n4" podUID="bc5be61d-7ca6-4860-ac6d-92cf4d56ac51"
Aug 05 17:12:56.100062 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:56.099968    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fgk52" podUID="a16139f2-7cad-4290-9562-7b0d79df30f8"
Aug 05 17:12:56.100062 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.099993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:56.100160 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:56.100113    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pg7tl" podUID="1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa"
Aug 05 17:12:56.106839 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.106815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-tmp-dir\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.110428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.110383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vx2t\" (UniqueName: \"kubernetes.io/projected/3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826-kube-api-access-9vx2t\") pod \"node-resolver-2czsm\" (UID: \"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826\") " pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.144927 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.144908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2czsm"
Aug 05 17:12:56.174323 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:56.174159    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf4abb5_9d9e_4a41_9b89_d0c36b2c1826.slice/crio-4867b3b9864b9405965860ead328293f545f0d35b3a458564fbc3730350fbb32 WatchSource:0}: Error finding container 4867b3b9864b9405965860ead328293f545f0d35b3a458564fbc3730350fbb32: Status 404 returned error can't find the container with id 4867b3b9864b9405965860ead328293f545f0d35b3a458564fbc3730350fbb32
Aug 05 17:12:56.245374 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.245291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2czsm" event={"ID":"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826","Type":"ContainerStarted","Data":"4867b3b9864b9405965860ead328293f545f0d35b3a458564fbc3730350fbb32"}
Aug 05 17:12:56.748903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.748874    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeReady"
Aug 05 17:12:56.749048 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.749023    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 05 17:12:56.798476 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.798443    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"]
Aug 05 17:12:56.801812 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.801796    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66b94b8485-bqdrx"]
Aug 05 17:12:56.801922 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.801904    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:56.805291 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.805268    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xwpsq\""
Aug 05 17:12:56.805291 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.805265    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Aug 05 17:12:56.805503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.805482    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.805885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.805872    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.807154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.807133    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.807245 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.807228    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Aug 05 17:12:56.808453 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.808434    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 05 17:12:56.809014 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.808725    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 05 17:12:56.809014 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.808779    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9s4jc\""
Aug 05 17:12:56.809014 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.808725    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 05 17:12:56.817333 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.817286    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"]
Aug 05 17:12:56.818270 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.818251    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 05 17:12:56.818639 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.818620    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"]
Aug 05 17:12:56.824060 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.824038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:56.826752 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.826628    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"]
Aug 05 17:12:56.827707 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.827348    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Aug 05 17:12:56.827707 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.827561    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.827707 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.827609    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vtsz4\""
Aug 05 17:12:56.827980 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.827959    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.834138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.834119    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zf7p6"]
Aug 05 17:12:56.834247 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.834235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"
Aug 05 17:12:56.836589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.836572    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"]
Aug 05 17:12:56.836696 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.836666    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:56.837788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.837768    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.838172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.838153    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.838256 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.838185    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-fm28g\""
Aug 05 17:12:56.839063 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.839044    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 05 17:12:56.839285 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.839271    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 05 17:12:56.839438 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.839420    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mrwwg\""
Aug 05 17:12:56.842345 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.842327    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-97hsq"]
Aug 05 17:12:56.842442 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.842398    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:56.845032 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.845013    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Aug 05 17:12:56.846286 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.846265    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Aug 05 17:12:56.846542 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.846511    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.846693 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.846676    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fwcmz\""
Aug 05 17:12:56.846767 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.846716    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.850333 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.850298    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"]
Aug 05 17:12:56.852684 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.852666    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:56.857084 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.857059    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Aug 05 17:12:56.857410 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.857391    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.860956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.860219    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Aug 05 17:12:56.860956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.860779    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.860956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.860781    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9vfgf\""
Aug 05 17:12:56.865075 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.864358    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77679b9596-fw5nw"]
Aug 05 17:12:56.865075 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.864690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:56.866435 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.866417    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Aug 05 17:12:56.868491 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.868472    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Aug 05 17:12:56.868848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.868522    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hknps\""
Aug 05 17:12:56.869121 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.868620    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Aug 05 17:12:56.872794 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.872776    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"]
Aug 05 17:12:56.873167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.873153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:56.876481 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.876460    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.876678 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.876657    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"]
Aug 05 17:12:56.876768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.876743    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"
Aug 05 17:12:56.877107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.877077    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Aug 05 17:12:56.877107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.877099    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Aug 05 17:12:56.877241 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.877110    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.877241 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.877136    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Aug 05 17:12:56.877241 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.877143    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Aug 05 17:12:56.877422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.877263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l5nxj\""
Aug 05 17:12:56.879250 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.879232    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.881791 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.881638    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.881888 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.881832    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dp2vn\""
Aug 05 17:12:56.882885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.882854    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-5tj5j"]
Aug 05 17:12:56.882977 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.882934    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:56.886262 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.886219    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.886595 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.886577    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cgrmq\""
Aug 05 17:12:56.886757 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.886741    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Aug 05 17:12:56.886818 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.886800    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.887585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.887546    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Aug 05 17:12:56.894963 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.894944    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dgm48"]
Aug 05 17:12:56.895070 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.895061    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:56.899706 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.899651    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Aug 05 17:12:56.900010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.899993    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Aug 05 17:12:56.900233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.900013    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.900399 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.900241    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.900399 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.900368    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-t6xd7\""
Aug 05 17:12:56.902717 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-certificates\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.902717 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4sxx\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-kube-api-access-k4sxx\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.902717 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902689    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-trusted-ca\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902759    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-installation-pull-secrets\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-bound-sa-token\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02bfa378-627d-499b-84c9-eb1c9f19a574-ca-trust-extracted\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902967    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-image-registry-private-configuration\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:56.903000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.902993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vn2g\" (UniqueName: \"kubernetes.io/projected/4effde24-1170-4722-97ff-dda58d6a09b7-kube-api-access-5vn2g\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:56.903410 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.903018    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4effde24-1170-4722-97ff-dda58d6a09b7-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:56.904978 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.904960    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b94b8485-bqdrx"]
Aug 05 17:12:56.905110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.904996    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"]
Aug 05 17:12:56.905110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905025    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"]
Aug 05 17:12:56.905110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905046    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-97hsq"]
Aug 05 17:12:56.905110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905065    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"]
Aug 05 17:12:56.905110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905083    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"]
Aug 05 17:12:56.905110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905104    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"]
Aug 05 17:12:56.905523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905123    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77679b9596-fw5nw"]
Aug 05 17:12:56.905523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905142    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dgm48"]
Aug 05 17:12:56.905523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905158    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:56.905523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905166    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-5tj5j"]
Aug 05 17:12:56.905523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zf7p6"]
Aug 05 17:12:56.905523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.905214    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"]
Aug 05 17:12:56.906915 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.906900    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Aug 05 17:12:56.907755 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.907738    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 05 17:12:56.909597 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.909579    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 05 17:12:56.909687 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.909601    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 05 17:12:56.910039 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:56.909887    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tvn52\""
Aug 05 17:12:57.003495 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003421    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprqc\" (UniqueName: \"kubernetes.io/projected/b9544517-0371-4225-95c0-3f7c64e68bed-kube-api-access-vprqc\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.003495 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9aa898-2769-4d57-9eac-c585ad07a01a-tmp\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.003495 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-default-certificate\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003583    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-config-volume\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.003630    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-serving-cert\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4effde24-1170-4722-97ff-dda58d6a09b7-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.003705    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls podName:4effde24-1170-4722-97ff-dda58d6a09b7 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.503690252 +0000 UTC m=+36.036874475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-ph4zq" (UID: "4effde24-1170-4722-97ff-dda58d6a09b7") : secret "cluster-monitoring-operator-tls" not found
Aug 05 17:12:57.003749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-config\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfgn\" (UniqueName: \"kubernetes.io/projected/290d541c-fa30-4f67-b8ba-0360072ef692-kube-api-access-8cfgn\") pod \"ingress-canary-dgm48\" (UID: \"290d541c-fa30-4f67-b8ba-0360072ef692\") " pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78c6s\" (UniqueName: \"kubernetes.io/projected/08dd3338-c892-4d6a-a10b-f7284ce37dcd-kube-api-access-78c6s\") pod \"network-check-source-65bdbbff45-j22bh\" (UID: \"08dd3338-c892-4d6a-a10b-f7284ce37dcd\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003834    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t4xw\" (UniqueName: \"kubernetes.io/projected/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-kube-api-access-7t4xw\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.003942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-installation-pull-secrets\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-bound-sa-token\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.004138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-stats-auth\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02bfa378-627d-499b-84c9-eb1c9f19a574-ca-trust-extracted\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-tmp-dir\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm426\" (UniqueName: \"kubernetes.io/projected/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-kube-api-access-dm426\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q6xf\" (UniqueName: \"kubernetes.io/projected/cf459bdb-e018-49c8-9411-6533b5ea871b-kube-api-access-6q6xf\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9544517-0371-4225-95c0-3f7c64e68bed-trusted-ca\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004509    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4effde24-1170-4722-97ff-dda58d6a09b7-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9hw\" (UniqueName: \"kubernetes.io/projected/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-kube-api-access-gf9hw\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.004564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004566    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9544517-0371-4225-95c0-3f7c64e68bed-config\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02bfa378-627d-499b-84c9-eb1c9f19a574-ca-trust-extracted\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.004598    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.004612    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b94b8485-bqdrx: secret "image-registry-tls" not found
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.004666    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls podName:02bfa378-627d-499b-84c9-eb1c9f19a574 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.50465049 +0000 UTC m=+36.037834713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls") pod "image-registry-66b94b8485-bqdrx" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574") : secret "image-registry-tls" not found
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9544517-0371-4225-95c0-3f7c64e68bed-serving-cert\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-image-registry-private-configuration\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vn2g\" (UniqueName: \"kubernetes.io/projected/4effde24-1170-4722-97ff-dda58d6a09b7-kube-api-access-5vn2g\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0442c145-eda8-402a-9e11-4ee7c7f1237c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004768    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004893    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9aa898-2769-4d57-9eac-c585ad07a01a-trusted-ca-bundle\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-certificates\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4sxx\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-kube-api-access-k4sxx\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.004999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9aa898-2769-4d57-9eac-c585ad07a01a-service-ca-bundle\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.005023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-trusted-ca\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/290d541c-fa30-4f67-b8ba-0360072ef692-cert\") pod \"ingress-canary-dgm48\" (UID: \"290d541c-fa30-4f67-b8ba-0360072ef692\") " pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f9aa898-2769-4d57-9eac-c585ad07a01a-serving-cert\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cczn4\" (UniqueName: \"kubernetes.io/projected/8f9aa898-2769-4d57-9eac-c585ad07a01a-kube-api-access-cczn4\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8x4\" (UniqueName: \"kubernetes.io/projected/58c51684-7e60-44ef-a39d-ac0474c834dd-kube-api-access-6q8x4\") pod \"volume-data-source-validator-5b8776669-ft2lq\" (UID: \"58c51684-7e60-44ef-a39d-ac0474c834dd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f9aa898-2769-4d57-9eac-c585ad07a01a-snapshots\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005190    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc9ls\" (UniqueName: \"kubernetes.io/projected/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-kube-api-access-lc9ls\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.005480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.005717 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.005480    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-certificates\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.006111 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.006093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-trusted-ca\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.007069 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.007038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-installation-pull-secrets\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.007554 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.007536    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-image-registry-private-configuration\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.020034 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.020009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4sxx\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-kube-api-access-k4sxx\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.021079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.021057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-bound-sa-token\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.021167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.021061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vn2g\" (UniqueName: \"kubernetes.io/projected/4effde24-1170-4722-97ff-dda58d6a09b7-kube-api-access-5vn2g\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:57.106187 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-config\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.106187 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfgn\" (UniqueName: \"kubernetes.io/projected/290d541c-fa30-4f67-b8ba-0360072ef692-kube-api-access-8cfgn\") pod \"ingress-canary-dgm48\" (UID: \"290d541c-fa30-4f67-b8ba-0360072ef692\") " pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.106445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106226    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78c6s\" (UniqueName: \"kubernetes.io/projected/08dd3338-c892-4d6a-a10b-f7284ce37dcd-kube-api-access-78c6s\") pod \"network-check-source-65bdbbff45-j22bh\" (UID: \"08dd3338-c892-4d6a-a10b-f7284ce37dcd\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"
Aug 05 17:12:57.106445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t4xw\" (UniqueName: \"kubernetes.io/projected/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-kube-api-access-7t4xw\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.106445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106283    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.106445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.106445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.106445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-stats-auth\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106467    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-tmp-dir\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm426\" (UniqueName: \"kubernetes.io/projected/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-kube-api-access-dm426\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106539    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls podName:97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.606520139 +0000 UTC m=+36.139704376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls") pod "dns-default-zf7p6" (UID: "97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d") : secret "dns-default-metrics-tls" not found
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106583    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.606573055 +0000 UTC m=+36.139757279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : configmap references non-existent config key: service-ca.crt
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106607    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106631    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106655    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls podName:cf459bdb-e018-49c8-9411-6533b5ea871b nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.606643853 +0000 UTC m=+36.139828093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9ftmg" (UID: "cf459bdb-e018-49c8-9411-6533b5ea871b") : secret "samples-operator-tls" not found
Aug 05 17:12:57.106659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q6xf\" (UniqueName: \"kubernetes.io/projected/cf459bdb-e018-49c8-9411-6533b5ea871b-kube-api-access-6q6xf\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.106680    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.606662844 +0000 UTC m=+36.139847084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : secret "router-metrics-certs-default" not found
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9544517-0371-4225-95c0-3f7c64e68bed-trusted-ca\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9hw\" (UniqueName: \"kubernetes.io/projected/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-kube-api-access-gf9hw\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-config\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9544517-0371-4225-95c0-3f7c64e68bed-config\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9544517-0371-4225-95c0-3f7c64e68bed-serving-cert\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0442c145-eda8-402a-9e11-4ee7c7f1237c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9aa898-2769-4d57-9eac-c585ad07a01a-trusted-ca-bundle\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-tmp-dir\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.106996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9aa898-2769-4d57-9eac-c585ad07a01a-service-ca-bundle\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/290d541c-fa30-4f67-b8ba-0360072ef692-cert\") pod \"ingress-canary-dgm48\" (UID: \"290d541c-fa30-4f67-b8ba-0360072ef692\") " pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.107042    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f9aa898-2769-4d57-9eac-c585ad07a01a-serving-cert\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.107078 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.107083    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert podName:0442c145-eda8-402a-9e11-4ee7c7f1237c nodeName:}" failed. No retries permitted until 2026-08-05 17:12:57.607070874 +0000 UTC m=+36.140255105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-2mnvv" (UID: "0442c145-eda8-402a-9e11-4ee7c7f1237c") : secret "networking-console-plugin-cert" not found
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cczn4\" (UniqueName: \"kubernetes.io/projected/8f9aa898-2769-4d57-9eac-c585ad07a01a-kube-api-access-cczn4\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8x4\" (UniqueName: \"kubernetes.io/projected/58c51684-7e60-44ef-a39d-ac0474c834dd-kube-api-access-6q8x4\") pod \"volume-data-source-validator-5b8776669-ft2lq\" (UID: \"58c51684-7e60-44ef-a39d-ac0474c834dd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f9aa898-2769-4d57-9eac-c585ad07a01a-snapshots\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc9ls\" (UniqueName: \"kubernetes.io/projected/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-kube-api-access-lc9ls\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprqc\" (UniqueName: \"kubernetes.io/projected/b9544517-0371-4225-95c0-3f7c64e68bed-kube-api-access-vprqc\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9aa898-2769-4d57-9eac-c585ad07a01a-tmp\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-default-certificate\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-config-volume\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-serving-cert\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9544517-0371-4225-95c0-3f7c64e68bed-config\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.107833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9544517-0371-4225-95c0-3f7c64e68bed-trusted-ca\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.108472 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.107919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9aa898-2769-4d57-9eac-c585ad07a01a-trusted-ca-bundle\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.108472 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.108070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f9aa898-2769-4d57-9eac-c585ad07a01a-service-ca-bundle\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.108472 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.108379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f9aa898-2769-4d57-9eac-c585ad07a01a-tmp\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.108843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.108625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.108966 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.108944    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f9aa898-2769-4d57-9eac-c585ad07a01a-snapshots\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.109030 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.108986    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0442c145-eda8-402a-9e11-4ee7c7f1237c-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:57.109617 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.109483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-config-volume\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.109698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.109650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9544517-0371-4225-95c0-3f7c64e68bed-serving-cert\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.110033 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.109962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-stats-auth\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.110033 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.109968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-serving-cert\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.110555 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.110517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f9aa898-2769-4d57-9eac-c585ad07a01a-serving-cert\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.111543 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.111520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.111649 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.111526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-default-certificate\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.111649 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.111590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/290d541c-fa30-4f67-b8ba-0360072ef692-cert\") pod \"ingress-canary-dgm48\" (UID: \"290d541c-fa30-4f67-b8ba-0360072ef692\") " pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.119045 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.118652    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfgn\" (UniqueName: \"kubernetes.io/projected/290d541c-fa30-4f67-b8ba-0360072ef692-kube-api-access-8cfgn\") pod \"ingress-canary-dgm48\" (UID: \"290d541c-fa30-4f67-b8ba-0360072ef692\") " pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.119045 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.118884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t4xw\" (UniqueName: \"kubernetes.io/projected/df0256bf-e00b-43ec-96ef-ad9e6f2717c9-kube-api-access-7t4xw\") pod \"kube-storage-version-migrator-operator-7878f4b88c-q5wr7\" (UID: \"df0256bf-e00b-43ec-96ef-ad9e6f2717c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.119953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.119883    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8x4\" (UniqueName: \"kubernetes.io/projected/58c51684-7e60-44ef-a39d-ac0474c834dd-kube-api-access-6q8x4\") pod \"volume-data-source-validator-5b8776669-ft2lq\" (UID: \"58c51684-7e60-44ef-a39d-ac0474c834dd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"
Aug 05 17:12:57.120503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.120068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q6xf\" (UniqueName: \"kubernetes.io/projected/cf459bdb-e018-49c8-9411-6533b5ea871b-kube-api-access-6q6xf\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:57.120503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.120461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm426\" (UniqueName: \"kubernetes.io/projected/02c0d02c-18e8-456c-a4d0-2af2ad30c7ed-kube-api-access-dm426\") pod \"service-ca-operator-784f97bcf6-gm767\" (UID: \"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.120869 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.120837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprqc\" (UniqueName: \"kubernetes.io/projected/b9544517-0371-4225-95c0-3f7c64e68bed-kube-api-access-vprqc\") pod \"console-operator-5ccd6b445f-5tj5j\" (UID: \"b9544517-0371-4225-95c0-3f7c64e68bed\") " pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.121483 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.121454    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cczn4\" (UniqueName: \"kubernetes.io/projected/8f9aa898-2769-4d57-9eac-c585ad07a01a-kube-api-access-cczn4\") pod \"insights-operator-77696685b-97hsq\" (UID: \"8f9aa898-2769-4d57-9eac-c585ad07a01a\") " pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.121675 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.121654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9hw\" (UniqueName: \"kubernetes.io/projected/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-kube-api-access-gf9hw\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.122042 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.122014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc9ls\" (UniqueName: \"kubernetes.io/projected/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-kube-api-access-lc9ls\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.122461 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.122429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78c6s\" (UniqueName: \"kubernetes.io/projected/08dd3338-c892-4d6a-a10b-f7284ce37dcd-kube-api-access-78c6s\") pod \"network-check-source-65bdbbff45-j22bh\" (UID: \"08dd3338-c892-4d6a-a10b-f7284ce37dcd\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"
Aug 05 17:12:57.154766 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.154747    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"
Aug 05 17:12:57.174052 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.174033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"
Aug 05 17:12:57.181869 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.181848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-97hsq"
Aug 05 17:12:57.204427 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.204394    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"
Aug 05 17:12:57.212732 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.212710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"
Aug 05 17:12:57.219933 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.219875    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:12:57.228057 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.228021    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dgm48"
Aug 05 17:12:57.250187 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.250093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2czsm" event={"ID":"3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826","Type":"ContainerStarted","Data":"2a0a3fbda16ae5662704aab423caea426991db21a466c06f8d8bc7208b5ebd88"}
Aug 05 17:12:57.273567 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.273513    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2czsm" podStartSLOduration=2.273492326 podStartE2EDuration="2.273492326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:57.272757275 +0000 UTC m=+35.805941522" watchObservedRunningTime="2026-08-05 17:12:57.273492326 +0000 UTC m=+35.806676577"
Aug 05 17:12:57.475849 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.474904    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-97hsq"]
Aug 05 17:12:57.477291 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.477111    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767"]
Aug 05 17:12:57.481815 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.481773    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9aa898_2769_4d57_9eac_c585ad07a01a.slice/crio-11651f0860c90a6d27066101eac725c1a15ce069729c1b3fa3205876cd21b4d6 WatchSource:0}: Error finding container 11651f0860c90a6d27066101eac725c1a15ce069729c1b3fa3205876cd21b4d6: Status 404 returned error can't find the container with id 11651f0860c90a6d27066101eac725c1a15ce069729c1b3fa3205876cd21b4d6
Aug 05 17:12:57.483528 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.483503    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq"]
Aug 05 17:12:57.485370 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.485331    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c51684_7e60_44ef_a39d_ac0474c834dd.slice/crio-5b9b4ba2c2510968ff6b01a55c9bbba4ec906e6af7e4219991b117af59649d19 WatchSource:0}: Error finding container 5b9b4ba2c2510968ff6b01a55c9bbba4ec906e6af7e4219991b117af59649d19: Status 404 returned error can't find the container with id 5b9b4ba2c2510968ff6b01a55c9bbba4ec906e6af7e4219991b117af59649d19
Aug 05 17:12:57.490144 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.489987    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c0d02c_18e8_456c_a4d0_2af2ad30c7ed.slice/crio-1e1fdadb20c16c2249e4b268e799415233fe05fdcf88f81faab439ba95b9a5b1 WatchSource:0}: Error finding container 1e1fdadb20c16c2249e4b268e799415233fe05fdcf88f81faab439ba95b9a5b1: Status 404 returned error can't find the container with id 1e1fdadb20c16c2249e4b268e799415233fe05fdcf88f81faab439ba95b9a5b1
Aug 05 17:12:57.515525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.515460    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh"]
Aug 05 17:12:57.517459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.517133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:57.517459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.517267    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:57.517837 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.517481    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:12:57.517837 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.517499    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b94b8485-bqdrx: secret "image-registry-tls" not found
Aug 05 17:12:57.517837 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.517555    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls podName:02bfa378-627d-499b-84c9-eb1c9f19a574 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.517535575 +0000 UTC m=+37.050719821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls") pod "image-registry-66b94b8485-bqdrx" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574") : secret "image-registry-tls" not found
Aug 05 17:12:57.518249 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.518157    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 05 17:12:57.518249 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.518222    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls podName:4effde24-1170-4722-97ff-dda58d6a09b7 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.518202311 +0000 UTC m=+37.051386536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-ph4zq" (UID: "4effde24-1170-4722-97ff-dda58d6a09b7") : secret "cluster-monitoring-operator-tls" not found
Aug 05 17:12:57.519032 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.519006    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dd3338_c892_4d6a_a10b_f7284ce37dcd.slice/crio-8d2bc3b5767aa0db8d6b2dd4629f0c406e55fd968d5d8de35583638f7f0a1cac WatchSource:0}: Error finding container 8d2bc3b5767aa0db8d6b2dd4629f0c406e55fd968d5d8de35583638f7f0a1cac: Status 404 returned error can't find the container with id 8d2bc3b5767aa0db8d6b2dd4629f0c406e55fd968d5d8de35583638f7f0a1cac
Aug 05 17:12:57.520095 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.520064    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dgm48"]
Aug 05 17:12:57.523079 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.523049    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290d541c_fa30_4f67_b8ba_0360072ef692.slice/crio-7f84c83141faaffabe5b57556ca2a8ebbf49076df9d0d9388487147689b1e52f WatchSource:0}: Error finding container 7f84c83141faaffabe5b57556ca2a8ebbf49076df9d0d9388487147689b1e52f: Status 404 returned error can't find the container with id 7f84c83141faaffabe5b57556ca2a8ebbf49076df9d0d9388487147689b1e52f
Aug 05 17:12:57.535789 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.535771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7"]
Aug 05 17:12:57.538562 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.538543    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0256bf_e00b_43ec_96ef_ad9e6f2717c9.slice/crio-48fbd4a5237955cb14e6ba2783f6e9f145bf5307f927dc3024ca22e445dc8485 WatchSource:0}: Error finding container 48fbd4a5237955cb14e6ba2783f6e9f145bf5307f927dc3024ca22e445dc8485: Status 404 returned error can't find the container with id 48fbd4a5237955cb14e6ba2783f6e9f145bf5307f927dc3024ca22e445dc8485
Aug 05 17:12:57.549558 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.549538    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-5tj5j"]
Aug 05 17:12:57.560534 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:12:57.560406    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9544517_0371_4225_95c0_3f7c64e68bed.slice/crio-786df468aa056ef315757fb2f7211961842b82fc00647dac730420bf7114f3b6 WatchSource:0}: Error finding container 786df468aa056ef315757fb2f7211961842b82fc00647dac730420bf7114f3b6: Status 404 returned error can't find the container with id 786df468aa056ef315757fb2f7211961842b82fc00647dac730420bf7114f3b6
Aug 05 17:12:57.618534 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.618499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.618673 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.618550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:57.618673 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.618609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:57.618850 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618682    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.618663013 +0000 UTC m=+37.151847251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : configmap references non-existent config key: service-ca.crt
Aug 05 17:12:57.618850 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.618741    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:57.618850 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:57.618769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:57.618850 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618769    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 05 17:12:57.618850 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618820    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:12:57.618850 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618832    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:12:57.619216 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618840    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls podName:cf459bdb-e018-49c8-9411-6533b5ea871b nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.61882981 +0000 UTC m=+37.152014036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9ftmg" (UID: "cf459bdb-e018-49c8-9411-6533b5ea871b") : secret "samples-operator-tls" not found
Aug 05 17:12:57.619216 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618874    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 05 17:12:57.619216 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618931    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert podName:0442c145-eda8-402a-9e11-4ee7c7f1237c nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.618904509 +0000 UTC m=+37.152088732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-2mnvv" (UID: "0442c145-eda8-402a-9e11-4ee7c7f1237c") : secret "networking-console-plugin-cert" not found
Aug 05 17:12:57.619216 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618959    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls podName:97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.618939358 +0000 UTC m=+37.152123585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls") pod "dns-default-zf7p6" (UID: "97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d") : secret "dns-default-metrics-tls" not found
Aug 05 17:12:57.619216 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:57.618973    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:58.618965576 +0000 UTC m=+37.152149804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : secret "router-metrics-certs-default" not found
Aug 05 17:12:58.100176 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.100139    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:12:58.100382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.100179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:12:58.100382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.100179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:12:58.103863 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.103846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:12:58.104363 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.104347    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4r2js\""
Aug 05 17:12:58.104466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.104411    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vwprk\""
Aug 05 17:12:58.104883 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.104867    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:12:58.160758 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.160724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:12:58.255094 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.255054    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh" event={"ID":"08dd3338-c892-4d6a-a10b-f7284ce37dcd","Type":"ContainerStarted","Data":"8d2bc3b5767aa0db8d6b2dd4629f0c406e55fd968d5d8de35583638f7f0a1cac"}
Aug 05 17:12:58.259228 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.259115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq" event={"ID":"58c51684-7e60-44ef-a39d-ac0474c834dd","Type":"ContainerStarted","Data":"5b9b4ba2c2510968ff6b01a55c9bbba4ec906e6af7e4219991b117af59649d19"}
Aug 05 17:12:58.261118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.261093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" event={"ID":"b9544517-0371-4225-95c0-3f7c64e68bed","Type":"ContainerStarted","Data":"786df468aa056ef315757fb2f7211961842b82fc00647dac730420bf7114f3b6"}
Aug 05 17:12:58.263754 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.263728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:12:58.267593 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.267513    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767" event={"ID":"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed","Type":"ContainerStarted","Data":"1e1fdadb20c16c2249e4b268e799415233fe05fdcf88f81faab439ba95b9a5b1"}
Aug 05 17:12:58.270579 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.270515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-97hsq" event={"ID":"8f9aa898-2769-4d57-9eac-c585ad07a01a","Type":"ContainerStarted","Data":"11651f0860c90a6d27066101eac725c1a15ce069729c1b3fa3205876cd21b4d6"}
Aug 05 17:12:58.272725 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.272653    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7" event={"ID":"df0256bf-e00b-43ec-96ef-ad9e6f2717c9","Type":"ContainerStarted","Data":"48fbd4a5237955cb14e6ba2783f6e9f145bf5307f927dc3024ca22e445dc8485"}
Aug 05 17:12:58.274684 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.274662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dgm48" event={"ID":"290d541c-fa30-4f67-b8ba-0360072ef692","Type":"ContainerStarted","Data":"7f84c83141faaffabe5b57556ca2a8ebbf49076df9d0d9388487147689b1e52f"}
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.527860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.527980    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.527993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.528056    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls podName:4effde24-1170-4722-97ff-dda58d6a09b7 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.528036591 +0000 UTC m=+39.061220829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-ph4zq" (UID: "4effde24-1170-4722-97ff-dda58d6a09b7") : secret "cluster-monitoring-operator-tls" not found
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.528164    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.528177    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b94b8485-bqdrx: secret "image-registry-tls" not found
Aug 05 17:12:58.528287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.528213    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls podName:02bfa378-627d-499b-84c9-eb1c9f19a574 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.528201091 +0000 UTC m=+39.061385316 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls") pod "image-registry-66b94b8485-bqdrx" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574") : secret "image-registry-tls" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.629073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.629161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.629277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.629305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:58.629381    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.629534    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.629514187 +0000 UTC m=+39.162698435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : configmap references non-existent config key: service-ca.crt
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.629957    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630010    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls podName:cf459bdb-e018-49c8-9411-6533b5ea871b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.62999515 +0000 UTC m=+39.163179374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9ftmg" (UID: "cf459bdb-e018-49c8-9411-6533b5ea871b") : secret "samples-operator-tls" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630087    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630122    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert podName:0442c145-eda8-402a-9e11-4ee7c7f1237c nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.630111488 +0000 UTC m=+39.163295727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-2mnvv" (UID: "0442c145-eda8-402a-9e11-4ee7c7f1237c") : secret "networking-console-plugin-cert" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630179    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630207    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls podName:97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.63019818 +0000 UTC m=+39.163382417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls") pod "dns-default-zf7p6" (UID: "97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d") : secret "dns-default-metrics-tls" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630273    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 05 17:12:58.630363 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:12:58.630308    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:00.630297335 +0000 UTC m=+39.163481559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : secret "router-metrics-certs-default" not found
Aug 05 17:12:59.118511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:59.118448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:12:59.236850 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:59.236777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:12:59.310669 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:12:59.310577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:00.057842 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.057804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:00.547802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.547769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:13:00.548272 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.547877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:00.548272 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.547991    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:13:00.548272 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.548006    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b94b8485-bqdrx: secret "image-registry-tls" not found
Aug 05 17:13:00.548272 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.548053    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls podName:02bfa378-627d-499b-84c9-eb1c9f19a574 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.548040029 +0000 UTC m=+43.081224255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls") pod "image-registry-66b94b8485-bqdrx" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574") : secret "image-registry-tls" not found
Aug 05 17:13:00.548524 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.548403    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 05 17:13:00.548524 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.548455    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls podName:4effde24-1170-4722-97ff-dda58d6a09b7 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.548440236 +0000 UTC m=+43.081624465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-ph4zq" (UID: "4effde24-1170-4722-97ff-dda58d6a09b7") : secret "cluster-monitoring-operator-tls" not found
Aug 05 17:13:00.648884 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.648819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:13:00.649078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.648944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:00.649078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.648974    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:00.649078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.649026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:00.649078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:00.649061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:13:00.649287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649179    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 05 17:13:00.649287 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649232    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls podName:cf459bdb-e018-49c8-9411-6533b5ea871b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.649215209 +0000 UTC m=+43.182399433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9ftmg" (UID: "cf459bdb-e018-49c8-9411-6533b5ea871b") : secret "samples-operator-tls" not found
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649432    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.649418367 +0000 UTC m=+43.182602616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : configmap references non-existent config key: service-ca.crt
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649439    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649499    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649504    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.649486304 +0000 UTC m=+43.182670540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : secret "router-metrics-certs-default" not found
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649548    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert podName:0442c145-eda8-402a-9e11-4ee7c7f1237c nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.649536792 +0000 UTC m=+43.182721024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-2mnvv" (UID: "0442c145-eda8-402a-9e11-4ee7c7f1237c") : secret "networking-console-plugin-cert" not found
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649571    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:00.649697 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:00.649621    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls podName:97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d nodeName:}" failed. No retries permitted until 2026-08-05 17:13:04.649605686 +0000 UTC m=+43.182789918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls") pod "dns-default-zf7p6" (UID: "97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:01.054060 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:01.054026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:01.153446 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:01.153414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:01.714914 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:01.714860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:01.888880 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:01.888828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:02.947104 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:02.947069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:03.015616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:03.015581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:04.583465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.583429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:04.583908 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.583544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:13:04.583908 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.583587    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:13:04.583908 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.583612    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b94b8485-bqdrx: secret "image-registry-tls" not found
Aug 05 17:13:04.583908 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.583674    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls podName:02bfa378-627d-499b-84c9-eb1c9f19a574 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.583654252 +0000 UTC m=+51.116838494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls") pod "image-registry-66b94b8485-bqdrx" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574") : secret "image-registry-tls" not found
Aug 05 17:13:04.583908 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.583723    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 05 17:13:04.583908 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.583795    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls podName:4effde24-1170-4722-97ff-dda58d6a09b7 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.583777691 +0000 UTC m=+51.116961933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-ph4zq" (UID: "4effde24-1170-4722-97ff-dda58d6a09b7") : secret "cluster-monitoring-operator-tls" not found
Aug 05 17:13:04.684574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.684537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:04.684721 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.684581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:04.684721 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.684694    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:04.684821 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.684734    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 05 17:13:04.684821 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.684765    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls podName:97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.684745004 +0000 UTC m=+51.217929253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls") pod "dns-default-zf7p6" (UID: "97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:04.684821 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.684786    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.684776938 +0000 UTC m=+51.217961168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : secret "router-metrics-certs-default" not found
Aug 05 17:13:04.684965 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.684835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:04.684965 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.684874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:13:04.684965 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:04.684939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:13:04.685113 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.685004    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.684986045 +0000 UTC m=+51.218170274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : configmap references non-existent config key: service-ca.crt
Aug 05 17:13:04.685113 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.685026    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:13:04.685113 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.685054    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert podName:0442c145-eda8-402a-9e11-4ee7c7f1237c nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.685045818 +0000 UTC m=+51.218230040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-2mnvv" (UID: "0442c145-eda8-402a-9e11-4ee7c7f1237c") : secret "networking-console-plugin-cert" not found
Aug 05 17:13:04.685113 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.685089    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 05 17:13:04.685283 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:04.685119    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls podName:cf459bdb-e018-49c8-9411-6533b5ea871b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:12.685110055 +0000 UTC m=+51.218294277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9ftmg" (UID: "cf459bdb-e018-49c8-9411-6533b5ea871b") : secret "samples-operator-tls" not found
Aug 05 17:13:05.573583 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:05.573543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:05.645817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:05.645747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:06.296636 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.296581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7" event={"ID":"df0256bf-e00b-43ec-96ef-ad9e6f2717c9","Type":"ContainerStarted","Data":"76e8bc81a79a1a5d8216885719067fa3b6fb4cf332d34f5e984c1aedd4f9eca1"}
Aug 05 17:13:06.299369 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.299340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq" event={"ID":"58c51684-7e60-44ef-a39d-ac0474c834dd","Type":"ContainerStarted","Data":"40569039be180cd87aad7a0d1d2c2228672a20b5c6eda1e9c519325ba5d8ad7b"}
Aug 05 17:13:06.302776 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.302755    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerStarted","Data":"a7f22043c202f2db93a8085c91fd85f53d306bc76392eee97196f3e15914e461"}
Aug 05 17:13:06.305300 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.305275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767" event={"ID":"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed","Type":"ContainerStarted","Data":"5f3c0455b3a53324e211ae0966fb95bbf80dbbf8a0ca5bdf39510bca39154e2d"}
Aug 05 17:13:06.310378 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.310259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-97hsq" event={"ID":"8f9aa898-2769-4d57-9eac-c585ad07a01a","Type":"ContainerStarted","Data":"da3eb6b37b4b09548ef7bd4f30e3d5b5cae1c3bf23bb9860a0c8f8a23a4aa127"}
Aug 05 17:13:06.317167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.316407    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7" podStartSLOduration=28.810921897 podStartE2EDuration="37.31639076s" totalImagesPullingTime="8.505468863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:06.315769843 +0000 UTC m=+44.848954089" watchObservedRunningTime="2026-08-05 17:13:06.31639076 +0000 UTC m=+44.849575007"
Aug 05 17:13:06.364586 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.364526    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-ft2lq" podStartSLOduration=28.813836848 podStartE2EDuration="37.364506129s" totalImagesPullingTime="8.550669281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:06.363280583 +0000 UTC m=+44.896464829" watchObservedRunningTime="2026-08-05 17:13:06.364506129 +0000 UTC m=+44.897690376"
Aug 05 17:13:06.386149 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.386055    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767" podStartSLOduration=29.59078178 podStartE2EDuration="37.386040197s" totalImagesPullingTime="7.795258417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:06.385851067 +0000 UTC m=+44.919035312" watchObservedRunningTime="2026-08-05 17:13:06.386040197 +0000 UTC m=+44.919224442"
Aug 05 17:13:06.408285 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.408235    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-97hsq" podStartSLOduration=28.846147902 podStartE2EDuration="37.408214724s" totalImagesPullingTime="8.562066822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:06.408030138 +0000 UTC m=+44.941214407" watchObservedRunningTime="2026-08-05 17:13:06.408214724 +0000 UTC m=+44.941398982"
Aug 05 17:13:06.516550 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.516472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:06.580442 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:06.580367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:07.315808 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.315777    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/0.log"
Aug 05 17:13:07.316380 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.315822    2562 generic.go:356] "Generic (PLEG): container finished" podID="b9544517-0371-4225-95c0-3f7c64e68bed" containerID="c616ad1fb1a19fc6c96f34aac17e7f58ac3dcc6b3ea2221cbc8de29c53238d0f" exitCode=255
Aug 05 17:13:07.316380 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.315915    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" event={"ID":"b9544517-0371-4225-95c0-3f7c64e68bed","Type":"ContainerDied","Data":"c616ad1fb1a19fc6c96f34aac17e7f58ac3dcc6b3ea2221cbc8de29c53238d0f"}
Aug 05 17:13:07.316380 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.316133    2562 scope.go:117] "RemoveContainer" containerID="c616ad1fb1a19fc6c96f34aac17e7f58ac3dcc6b3ea2221cbc8de29c53238d0f"
Aug 05 17:13:07.318118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.318033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dgm48" event={"ID":"290d541c-fa30-4f67-b8ba-0360072ef692","Type":"ContainerStarted","Data":"9cbfe15c542767eee553091433072e7d67ee79c882b30f71b3eb9be57fcb1057"}
Aug 05 17:13:07.319508 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.319461    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh" event={"ID":"08dd3338-c892-4d6a-a10b-f7284ce37dcd","Type":"ContainerStarted","Data":"fd8b3225b6bf8d0c0ce531edba6afc69585a3554933bf9359d4e7b58705dd742"}
Aug 05 17:13:07.322453 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.322430    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d27b52c-4637-4d6c-b323-4da630c5a6b0" containerID="a7f22043c202f2db93a8085c91fd85f53d306bc76392eee97196f3e15914e461" exitCode=0
Aug 05 17:13:07.323293 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.323272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerDied","Data":"a7f22043c202f2db93a8085c91fd85f53d306bc76392eee97196f3e15914e461"}
Aug 05 17:13:07.379483 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.379424    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dgm48" podStartSLOduration=2.5595748130000002 podStartE2EDuration="11.379407167s" totalImagesPullingTime="8.819832354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:07.377830043 +0000 UTC m=+45.911014289" watchObservedRunningTime="2026-08-05 17:13:07.379407167 +0000 UTC m=+45.912591407"
Aug 05 17:13:07.380219 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.380177    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-j22bh" podStartSLOduration=25.855200485 podStartE2EDuration="34.380166712s" totalImagesPullingTime="8.524966227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:07.356402366 +0000 UTC m=+45.889586613" watchObservedRunningTime="2026-08-05 17:13:07.380166712 +0000 UTC m=+45.913350958"
Aug 05 17:13:07.849430 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.849394    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4wxcg"]
Aug 05 17:13:07.870042 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.869961    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:07.870551 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.869958    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4wxcg"]
Aug 05 17:13:07.872903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.872684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 05 17:13:07.873614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.873135    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m4zv5\""
Aug 05 17:13:07.873614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.873364    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 05 17:13:07.874594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.874260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:07.934280 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.934246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:07.934461 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.934299    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfncf\" (UniqueName: \"kubernetes.io/projected/a49fae96-7a78-4f89-b998-976654a7ae49-kube-api-access-hfncf\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:07.934461 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.934405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a49fae96-7a78-4f89-b998-976654a7ae49-crio-socket\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:07.934461 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.934447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49fae96-7a78-4f89-b998-976654a7ae49-data-volume\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:07.934602 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:07.934497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035115 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfncf\" (UniqueName: \"kubernetes.io/projected/a49fae96-7a78-4f89-b998-976654a7ae49-kube-api-access-hfncf\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035294 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a49fae96-7a78-4f89-b998-976654a7ae49-crio-socket\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035294 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49fae96-7a78-4f89-b998-976654a7ae49-data-volume\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035432 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035432 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a49fae96-7a78-4f89-b998-976654a7ae49-crio-socket\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035545 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.035438    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:08.035545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.035545 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.035498    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:08.535479396 +0000 UTC m=+47.068663622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:08.035714 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.035579    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:08.035714 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.035634    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:08.535620421 +0000 UTC m=+47.068804648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:08.035838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.035802    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49fae96-7a78-4f89-b998-976654a7ae49-data-volume\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.052630 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.052595    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfncf\" (UniqueName: \"kubernetes.io/projected/a49fae96-7a78-4f89-b998-976654a7ae49-kube-api-access-hfncf\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.067703 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.067675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:08.137245 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.137159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:08.192440 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.192409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:08.327397 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.327369    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:13:08.327783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.327767    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/0.log"
Aug 05 17:13:08.327838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.327802    2562 generic.go:356] "Generic (PLEG): container finished" podID="b9544517-0371-4225-95c0-3f7c64e68bed" containerID="5910bf35765f04ed1fd50ee3b24128a096b106c7918fd92c793cb1ada1a31e12" exitCode=255
Aug 05 17:13:08.327903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.327884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" event={"ID":"b9544517-0371-4225-95c0-3f7c64e68bed","Type":"ContainerDied","Data":"5910bf35765f04ed1fd50ee3b24128a096b106c7918fd92c793cb1ada1a31e12"}
Aug 05 17:13:08.331472 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.331449    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d27b52c-4637-4d6c-b323-4da630c5a6b0" containerID="716a8d53ad6d094f3edb082b37b454a89aa2ee3602c89aafeaadf7d69d463778" exitCode=0
Aug 05 17:13:08.331594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.331528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerDied","Data":"716a8d53ad6d094f3edb082b37b454a89aa2ee3602c89aafeaadf7d69d463778"}
Aug 05 17:13:08.337390 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.337370    2562 scope.go:117] "RemoveContainer" containerID="c616ad1fb1a19fc6c96f34aac17e7f58ac3dcc6b3ea2221cbc8de29c53238d0f"
Aug 05 17:13:08.337510 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.337493    2562 scope.go:117] "RemoveContainer" containerID="5910bf35765f04ed1fd50ee3b24128a096b106c7918fd92c793cb1ada1a31e12"
Aug 05 17:13:08.337703 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.337677    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-5tj5j_openshift-console-operator(b9544517-0371-4225-95c0-3f7c64e68bed)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" podUID="b9544517-0371-4225-95c0-3f7c64e68bed"
Aug 05 17:13:08.488947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.488918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:08.540712 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.540683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.540825 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.540808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:08.540878 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.540839    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:08.540914 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.540900    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:09.540880781 +0000 UTC m=+48.074065006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:08.540978 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.540950    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:08.541054 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:08.541039    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:09.54101944 +0000 UTC m=+48.074203665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:08.568013 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.567990    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:08.726168 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.726142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:08.839191 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:08.839163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:09.194208 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.194176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:09.201690 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.201668    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2czsm_3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826/dns-node-resolver/0.log"
Aug 05 17:13:09.299128 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.299097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:09.338804 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.338778    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" event={"ID":"8d27b52c-4637-4d6c-b323-4da630c5a6b0","Type":"ContainerStarted","Data":"68d1c25c2afd2cab974fbca97be73beb1c652a5fd293c5d73787f4e2b0174b08"}
Aug 05 17:13:09.340211 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.340193    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:13:09.340481 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.340468    2562 scope.go:117] "RemoveContainer" containerID="5910bf35765f04ed1fd50ee3b24128a096b106c7918fd92c793cb1ada1a31e12"
Aug 05 17:13:09.340611 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:09.340595    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-5tj5j_openshift-console-operator(b9544517-0371-4225-95c0-3f7c64e68bed)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" podUID="b9544517-0371-4225-95c0-3f7c64e68bed"
Aug 05 17:13:09.364027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.363989    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fw2rc" podStartSLOduration=11.569119589 podStartE2EDuration="47.363975236s" totalImagesPullingTime="35.794855647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:09.362459257 +0000 UTC m=+47.895643502" watchObservedRunningTime="2026-08-05 17:13:09.363975236 +0000 UTC m=+47.897159482"
Aug 05 17:13:09.478173 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.478085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:09.547919 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.547893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:09.548084 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.547983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:09.548084 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:09.548021    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:09.548084 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:09.548074    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:09.548241 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:09.548110    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:11.548088451 +0000 UTC m=+50.081272674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:09.548241 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:09.548132    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:11.548120456 +0000 UTC m=+50.081304687 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:09.577964 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.577938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:09.679764 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:09.679735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:10.205008 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:10.204982    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-69j8c_f58e0111-2bdd-4b12-8da2-d65cbf3de0b5/node-ca/0.log"
Aug 05 17:13:10.902841 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:10.902808    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:11.012500 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:11.012468    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dgm48_290d541c-fa30-4f67-b8ba-0360072ef692/serve-healthcheck-canary/0.log"
Aug 05 17:13:11.107037 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:11.107006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:11.204242 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:11.204210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:11.564690 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:11.564607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:11.564860 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:11.564724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:11.564860 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:11.564732    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:11.564860 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:11.564790    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:15.564775235 +0000 UTC m=+54.097959458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:11.565017 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:11.564873    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:11.565017 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:11.564941    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:15.564924716 +0000 UTC m=+54.098108962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:11.872406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:11.872309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:12.673984 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.673943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:13:12.674423 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.674097    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Aug 05 17:13:12.674423 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.674119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:12.674423 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.674164    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls podName:4effde24-1170-4722-97ff-dda58d6a09b7 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.674146755 +0000 UTC m=+67.207330978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-ph4zq" (UID: "4effde24-1170-4722-97ff-dda58d6a09b7") : secret "cluster-monitoring-operator-tls" not found
Aug 05 17:13:12.674423 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.674210    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 05 17:13:12.674423 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.674220    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b94b8485-bqdrx: secret "image-registry-tls" not found
Aug 05 17:13:12.674423 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.674262    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls podName:02bfa378-627d-499b-84c9-eb1c9f19a574 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.674249245 +0000 UTC m=+67.207433467 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls") pod "image-registry-66b94b8485-bqdrx" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574") : secret "image-registry-tls" not found
Aug 05 17:13:12.775300 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.775262    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:12.775300 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.775300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.775369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.775391    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775425    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.775441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775494    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775523    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775532    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:13:12.775536 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775498    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls podName:97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.775478126 +0000 UTC m=+67.308662353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls") pod "dns-default-zf7p6" (UID: "97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:12.775804 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775562    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.775549166 +0000 UTC m=+67.308733388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : configmap references non-existent config key: service-ca.crt
Aug 05 17:13:12.775804 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775580    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls podName:cf459bdb-e018-49c8-9411-6533b5ea871b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.775567159 +0000 UTC m=+67.308751386 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9ftmg" (UID: "cf459bdb-e018-49c8-9411-6533b5ea871b") : secret "samples-operator-tls" not found
Aug 05 17:13:12.775804 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775598    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs podName:b6e8bf52-e2e4-425a-bb29-46857e8f22f9 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.775588626 +0000 UTC m=+67.308772862 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs") pod "router-default-77679b9596-fw5nw" (UID: "b6e8bf52-e2e4-425a-bb29-46857e8f22f9") : secret "router-metrics-certs-default" not found
Aug 05 17:13:12.775804 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:12.775613    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert podName:0442c145-eda8-402a-9e11-4ee7c7f1237c nodeName:}" failed. No retries permitted until 2026-08-05 17:13:28.775604777 +0000 UTC m=+67.308789002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-2mnvv" (UID: "0442c145-eda8-402a-9e11-4ee7c7f1237c") : secret "networking-console-plugin-cert" not found
Aug 05 17:13:12.927558 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:12.927479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:15.381016 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.380985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:15.500256 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.500224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:15.599006 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.598977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:15.601161 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.601140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:15.601302 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:15.601288    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:15.601360 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.601305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:15.601397 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:15.601371    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls podName:a49fae96-7a78-4f89-b998-976654a7ae49 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:23.601356961 +0000 UTC m=+62.134541183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4wxcg" (UID: "a49fae96-7a78-4f89-b998-976654a7ae49") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:15.601739 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.601723    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a49fae96-7a78-4f89-b998-976654a7ae49-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:15.700768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.700738    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:15.803921 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.803891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:15.868528 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:15.868496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:16.185215 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:16.185137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:16.393479 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:16.393449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:16.577626 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:16.577591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:17.220279 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:17.220247    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:13:17.220459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:17.220299    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:13:17.220663 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:17.220650    2562 scope.go:117] "RemoveContainer" containerID="5910bf35765f04ed1fd50ee3b24128a096b106c7918fd92c793cb1ada1a31e12"
Aug 05 17:13:17.220807 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:17.220789    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-5tj5j_openshift-console-operator(b9544517-0371-4225-95c0-3f7c64e68bed)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" podUID="b9544517-0371-4225-95c0-3f7c64e68bed"
Aug 05 17:13:17.801133 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:17.801097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:17.981424 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:17.981383    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:18.553415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:18.553379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:18.625821 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:18.625790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:19.178214 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:19.178181    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:19.281235 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:19.281204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:20.217383 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:20.217351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:20.279623 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:20.279596    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:20.380159 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:20.380128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:20.611924 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:20.611850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:20.669217 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:20.669191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:20.780237 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:20.780213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:21.475009 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:21.474982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:21.574914 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:21.574891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:21.676362 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:21.676337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:21.776628 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:21.776555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:21.976027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:21.975987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:22.188838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:22.188764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:22.247933 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:22.247913    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mc8cp"
Aug 05 17:13:22.374700 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:22.374674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:22.590411 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:22.590384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:23.594679 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.594644    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:23.671896 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.671857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:23.674172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.674143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a49fae96-7a78-4f89-b998-976654a7ae49-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4wxcg\" (UID: \"a49fae96-7a78-4f89-b998-976654a7ae49\") " pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:23.782113 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.782090    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m4zv5\""
Aug 05 17:13:23.790531 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.790507    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4wxcg"
Aug 05 17:13:23.917866 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.917840    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4wxcg"]
Aug 05 17:13:23.921044 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:23.921020    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49fae96_7a78_4f89_b998_976654a7ae49.slice/crio-ba7dbffc6fdc460523b1309d226610a293618599e7d76e97c770467b0847e60c WatchSource:0}: Error finding container ba7dbffc6fdc460523b1309d226610a293618599e7d76e97c770467b0847e60c: Status 404 returned error can't find the container with id ba7dbffc6fdc460523b1309d226610a293618599e7d76e97c770467b0847e60c
Aug 05 17:13:23.985423 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:23.985394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:24.223279 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:24.223254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:24.289510 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:24.289481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:24.385534 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:24.385512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:24.393043 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:24.393022    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4wxcg" event={"ID":"a49fae96-7a78-4f89-b998-976654a7ae49","Type":"ContainerStarted","Data":"10a536ef71813653825a2a7f4d0ec42e9df5a5a833e66ca9543063a9c655ba9e"}
Aug 05 17:13:24.393154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:24.393075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4wxcg" event={"ID":"a49fae96-7a78-4f89-b998-976654a7ae49","Type":"ContainerStarted","Data":"ba7dbffc6fdc460523b1309d226610a293618599e7d76e97c770467b0847e60c"}
Aug 05 17:13:24.801292 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:24.801256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:25.386300 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:25.386270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:25.398024 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:25.397996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4wxcg" event={"ID":"a49fae96-7a78-4f89-b998-976654a7ae49","Type":"ContainerStarted","Data":"3289c75462f9f9aa7b0fbd4e0f275e17410d3a852eb7499977832c74518acc96"}
Aug 05 17:13:25.789220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:25.789185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:26.000547 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:26.000518    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:26.176407 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:26.176346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:26.405047 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:26.405013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4wxcg" event={"ID":"a49fae96-7a78-4f89-b998-976654a7ae49","Type":"ContainerStarted","Data":"b58e8e303e4cce4a269e864e2deb3ebedc9513cd9f2586b1f7e894872221da95"}
Aug 05 17:13:26.437902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:26.437808    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4wxcg" podStartSLOduration=17.590867727 podStartE2EDuration="19.437794235s" totalImagesPullingTime="1.846926508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:13:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:26.436001924 +0000 UTC m=+64.969186168" watchObservedRunningTime="2026-08-05 17:13:26.437794235 +0000 UTC m=+64.970978502"
Aug 05 17:13:27.705218 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.705171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:13:27.705218 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.705224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:13:27.709902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.709867    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:13:27.710043 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.710030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:13:27.718435 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.718411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc5be61d-7ca6-4860-ac6d-92cf4d56ac51-original-pull-secret\") pod \"global-pull-secret-syncer-452n4\" (UID: \"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51\") " pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:13:27.718521 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.718433    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa-metrics-certs\") pod \"network-metrics-daemon-pg7tl\" (UID: \"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa\") " pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:13:27.806478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.806449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:13:27.808721 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.808696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtn6t\" (UniqueName: \"kubernetes.io/projected/a16139f2-7cad-4290-9562-7b0d79df30f8-kube-api-access-vtn6t\") pod \"network-check-target-fgk52\" (UID: \"a16139f2-7cad-4290-9562-7b0d79df30f8\") " pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:13:27.834480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.834450    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-452n4"
Aug 05 17:13:27.847412 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.847389    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4r2js\""
Aug 05 17:13:27.854798 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.854769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:13:27.855577 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.855559    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vwprk\""
Aug 05 17:13:27.863209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.863071    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pg7tl"
Aug 05 17:13:27.988413 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:27.988371    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-452n4"]
Aug 05 17:13:27.991805 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:27.991769    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5be61d_7ca6_4860_ac6d_92cf4d56ac51.slice/crio-bd527e5b2aa4d3d434ce755b15d86c9c040614af2c657d149e66c590550e29ac WatchSource:0}: Error finding container bd527e5b2aa4d3d434ce755b15d86c9c040614af2c657d149e66c590550e29ac: Status 404 returned error can't find the container with id bd527e5b2aa4d3d434ce755b15d86c9c040614af2c657d149e66c590550e29ac
Aug 05 17:13:28.006741 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.006714    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fgk52"]
Aug 05 17:13:28.009558 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:28.009534    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16139f2_7cad_4290_9562_7b0d79df30f8.slice/crio-7b05f0f8c0f315f9cf5d111fc83c09d406d3be865999860e73ad597c3c0b36d5 WatchSource:0}: Error finding container 7b05f0f8c0f315f9cf5d111fc83c09d406d3be865999860e73ad597c3c0b36d5: Status 404 returned error can't find the container with id 7b05f0f8c0f315f9cf5d111fc83c09d406d3be865999860e73ad597c3c0b36d5
Aug 05 17:13:28.023941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.023920    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pg7tl"]
Aug 05 17:13:28.026830 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:28.026806    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8d2db2_6ec8_4e8e_9aa1_51560c53d2aa.slice/crio-1acea58aa88c29896ff04f4e96ab3531cbe5566c6aa11242484ac124a2ca9102 WatchSource:0}: Error finding container 1acea58aa88c29896ff04f4e96ab3531cbe5566c6aa11242484ac124a2ca9102: Status 404 returned error can't find the container with id 1acea58aa88c29896ff04f4e96ab3531cbe5566c6aa11242484ac124a2ca9102
Aug 05 17:13:28.411644 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.411558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pg7tl" event={"ID":"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa","Type":"ContainerStarted","Data":"1acea58aa88c29896ff04f4e96ab3531cbe5566c6aa11242484ac124a2ca9102"}
Aug 05 17:13:28.412978 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.412954    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fgk52" event={"ID":"a16139f2-7cad-4290-9562-7b0d79df30f8","Type":"ContainerStarted","Data":"c6a4c805a168038dcc2e71cbf1eb2a64c1a0db259b2c6cb02aaffcf11bea7f87"}
Aug 05 17:13:28.413104 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.412995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fgk52" event={"ID":"a16139f2-7cad-4290-9562-7b0d79df30f8","Type":"ContainerStarted","Data":"7b05f0f8c0f315f9cf5d111fc83c09d406d3be865999860e73ad597c3c0b36d5"}
Aug 05 17:13:28.413104 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.413040    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:13:28.414072 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.414053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-452n4" event={"ID":"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51","Type":"ContainerStarted","Data":"bd527e5b2aa4d3d434ce755b15d86c9c040614af2c657d149e66c590550e29ac"}
Aug 05 17:13:28.433412 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.433371    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fgk52" podStartSLOduration=66.433354501 podStartE2EDuration="1m6.433354501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:28.432435817 +0000 UTC m=+66.965620094" watchObservedRunningTime="2026-08-05 17:13:28.433354501 +0000 UTC m=+66.966538740"
Aug 05 17:13:28.713928 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.713891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:28.714403 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.714006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:13:28.716982 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.716956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"image-registry-66b94b8485-bqdrx\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") " pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:28.717100 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.716963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4effde24-1170-4722-97ff-dda58d6a09b7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-ph4zq\" (UID: \"4effde24-1170-4722-97ff-dda58d6a09b7\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:13:28.815134 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.815099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:28.815134 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.815134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:28.815353 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.815157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:28.815353 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.815273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:13:28.815353 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.815333    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:13:28.816017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.815990    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-service-ca-bundle\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:28.818118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.818087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d-metrics-tls\") pod \"dns-default-zf7p6\" (UID: \"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d\") " pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:28.818233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.818164    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6e8bf52-e2e4-425a-bb29-46857e8f22f9-metrics-certs\") pod \"router-default-77679b9596-fw5nw\" (UID: \"b6e8bf52-e2e4-425a-bb29-46857e8f22f9\") " pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:28.818233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.818170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf459bdb-e018-49c8-9411-6533b5ea871b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9ftmg\" (UID: \"cf459bdb-e018-49c8-9411-6533b5ea871b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:13:28.818468 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.818453    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0442c145-eda8-402a-9e11-4ee7c7f1237c-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-2mnvv\" (UID: \"0442c145-eda8-402a-9e11-4ee7c7f1237c\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:13:28.914429 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.914401    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xwpsq\""
Aug 05 17:13:28.922148 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.922122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"
Aug 05 17:13:28.922431 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.922411    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9s4jc\""
Aug 05 17:13:28.931406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.931372    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:28.938935 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.938782    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vtsz4\""
Aug 05 17:13:28.946507 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.946484    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"
Aug 05 17:13:28.964192 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.964091    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mrwwg\""
Aug 05 17:13:28.971594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.971309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:28.992107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.992084    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hknps\""
Aug 05 17:13:28.999593 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.999370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"
Aug 05 17:13:29.000193 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:28.999973    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l5nxj\""
Aug 05 17:13:29.009274 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.008178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:29.113877 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.111524    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq"]
Aug 05 17:13:29.151553 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.151477    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b94b8485-bqdrx"]
Aug 05 17:13:29.156235 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:29.156195    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bfa378_627d_499b_84c9_eb1c9f19a574.slice/crio-b2243266ce0212b36799d169c1bcc7c5493ec8b7ae7e80b3f3a7184d92938425 WatchSource:0}: Error finding container b2243266ce0212b36799d169c1bcc7c5493ec8b7ae7e80b3f3a7184d92938425: Status 404 returned error can't find the container with id b2243266ce0212b36799d169c1bcc7c5493ec8b7ae7e80b3f3a7184d92938425
Aug 05 17:13:29.177560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.177529    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg"]
Aug 05 17:13:29.193201 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.193166    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zf7p6"]
Aug 05 17:13:29.199493 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:29.199451    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fcfdfe_9e68_4ad1_84e6_d0f5368efa5d.slice/crio-43b0da7284ab187b7caa8fd318508e2d1e50b13ceccdc6fad7fd8cd25e3b3488 WatchSource:0}: Error finding container 43b0da7284ab187b7caa8fd318508e2d1e50b13ceccdc6fad7fd8cd25e3b3488: Status 404 returned error can't find the container with id 43b0da7284ab187b7caa8fd318508e2d1e50b13ceccdc6fad7fd8cd25e3b3488
Aug 05 17:13:29.422538 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.422494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx" event={"ID":"02bfa378-627d-499b-84c9-eb1c9f19a574","Type":"ContainerStarted","Data":"d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b"}
Aug 05 17:13:29.422720 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.422563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx" event={"ID":"02bfa378-627d-499b-84c9-eb1c9f19a574","Type":"ContainerStarted","Data":"b2243266ce0212b36799d169c1bcc7c5493ec8b7ae7e80b3f3a7184d92938425"}
Aug 05 17:13:29.422783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.422760    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:29.424413 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.424383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq" event={"ID":"4effde24-1170-4722-97ff-dda58d6a09b7","Type":"ContainerStarted","Data":"baeed0a1fdddea0c312aea8d79471aacc04e928706bc7b1d94e5580c7685ca3d"}
Aug 05 17:13:29.425822 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.425791    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zf7p6" event={"ID":"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d","Type":"ContainerStarted","Data":"43b0da7284ab187b7caa8fd318508e2d1e50b13ceccdc6fad7fd8cd25e3b3488"}
Aug 05 17:13:29.430189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.429458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg" event={"ID":"cf459bdb-e018-49c8-9411-6533b5ea871b","Type":"ContainerStarted","Data":"2a5954702f4ef9cab40b720e6013c1ae0318359601ceb5edba8c2d85e97d8e10"}
Aug 05 17:13:29.431424 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.431372    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv"]
Aug 05 17:13:29.432931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.432840    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77679b9596-fw5nw"]
Aug 05 17:13:29.436492 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:29.436464    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0442c145_eda8_402a_9e11_4ee7c7f1237c.slice/crio-5e432558f8115c3495285e2107f95e46489e6558805a759165a3a9f96548d6a5 WatchSource:0}: Error finding container 5e432558f8115c3495285e2107f95e46489e6558805a759165a3a9f96548d6a5: Status 404 returned error can't find the container with id 5e432558f8115c3495285e2107f95e46489e6558805a759165a3a9f96548d6a5
Aug 05 17:13:29.437759 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:29.437736    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e8bf52_e2e4_425a_bb29_46857e8f22f9.slice/crio-5088a748eb57b6930b86b81f7c9d646c10d556a247984edc3cfdc21432c634d0 WatchSource:0}: Error finding container 5088a748eb57b6930b86b81f7c9d646c10d556a247984edc3cfdc21432c634d0: Status 404 returned error can't find the container with id 5088a748eb57b6930b86b81f7c9d646c10d556a247984edc3cfdc21432c634d0
Aug 05 17:13:29.448552 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:29.448509    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx" podStartSLOduration=66.448494105 podStartE2EDuration="1m6.448494105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:29.446186114 +0000 UTC m=+67.979370360" watchObservedRunningTime="2026-08-05 17:13:29.448494105 +0000 UTC m=+67.981678332"
Aug 05 17:13:30.100045 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.099730    2562 scope.go:117] "RemoveContainer" containerID="5910bf35765f04ed1fd50ee3b24128a096b106c7918fd92c793cb1ada1a31e12"
Aug 05 17:13:30.439530 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.438656    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:13:30.439530 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.438756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" event={"ID":"b9544517-0371-4225-95c0-3f7c64e68bed","Type":"ContainerStarted","Data":"635490eda265c2804efeaa11c09bab2df1562fd266ab116b0a37c6147c8c9f60"}
Aug 05 17:13:30.439789 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.439595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:13:30.443006 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.442449    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pg7tl" event={"ID":"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa","Type":"ContainerStarted","Data":"46678301c89492778b20563d6e2a048e460f3645aac277b285ec7047fabc635b"}
Aug 05 17:13:30.443006 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.442500    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pg7tl" event={"ID":"1e8d2db2-6ec8-4e8e-9aa1-51560c53d2aa","Type":"ContainerStarted","Data":"72e698d8a3e332a7c5fb400a771e84d5a3ec5fefc65221a7e180e287fdd3e3aa"}
Aug 05 17:13:30.445418 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.445367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77679b9596-fw5nw" event={"ID":"b6e8bf52-e2e4-425a-bb29-46857e8f22f9","Type":"ContainerStarted","Data":"19d807536edf02e266e81647e3867b3c2a91d523061ffe2ae32e07ea6dcc157a"}
Aug 05 17:13:30.445418 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.445413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77679b9596-fw5nw" event={"ID":"b6e8bf52-e2e4-425a-bb29-46857e8f22f9","Type":"ContainerStarted","Data":"5088a748eb57b6930b86b81f7c9d646c10d556a247984edc3cfdc21432c634d0"}
Aug 05 17:13:30.447697 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.447642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv" event={"ID":"0442c145-eda8-402a-9e11-4ee7c7f1237c","Type":"ContainerStarted","Data":"5e432558f8115c3495285e2107f95e46489e6558805a759165a3a9f96548d6a5"}
Aug 05 17:13:30.487536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.487475    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77679b9596-fw5nw" podStartSLOduration=61.487454849 podStartE2EDuration="1m1.487454849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:30.486757169 +0000 UTC m=+69.019941409" watchObservedRunningTime="2026-08-05 17:13:30.487454849 +0000 UTC m=+69.020639095"
Aug 05 17:13:30.488340 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.488253    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" podStartSLOduration=53.007516681 podStartE2EDuration="1m1.488244458s" totalImagesPullingTime="8.480727777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:30.462389815 +0000 UTC m=+68.995574055" watchObservedRunningTime="2026-08-05 17:13:30.488244458 +0000 UTC m=+69.021428706"
Aug 05 17:13:30.507724 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:30.507646    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pg7tl" podStartSLOduration=66.487183876 podStartE2EDuration="1m8.507618624s" totalImagesPullingTime="2.020434748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:30.506723209 +0000 UTC m=+69.039907455" watchObservedRunningTime="2026-08-05 17:13:30.507618624 +0000 UTC m=+69.040802870"
Aug 05 17:13:31.009616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.009581    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:31.012753 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.012722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:31.439765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.439678    2562 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-5tj5j container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/readyz\": context deadline exceeded" start-of-body=
Aug 05 17:13:31.440213 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.439764    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j" podUID="b9544517-0371-4225-95c0-3f7c64e68bed" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.16:8443/readyz\": context deadline exceeded"
Aug 05 17:13:31.452025 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.451973    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:31.453343 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.453301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77679b9596-fw5nw"
Aug 05 17:13:31.536085 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.536053    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-5tj5j"
Aug 05 17:13:31.745931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.745901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:31.866439 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:31.866378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:32.036447 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:32.036350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:32.416970 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:32.416873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:32.483540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:32.483506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:32.817485 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:32.817450    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:32.884831 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:32.884792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:33.049533 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:33.049487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:33.146229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:33.146148    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:33.445304 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:33.445264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:33.619930 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:33.619892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:33.720461 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:33.720380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:34.464659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.464616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg" event={"ID":"cf459bdb-e018-49c8-9411-6533b5ea871b","Type":"ContainerStarted","Data":"516aca1d8f693bc19139b8bc3a408b1dba3ef97a940435fb49437bac1a43cd2b"}
Aug 05 17:13:34.464822 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.464685    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg" event={"ID":"cf459bdb-e018-49c8-9411-6533b5ea871b","Type":"ContainerStarted","Data":"5614d5c72451d47e2cb0fa65531d0150e31ee4b08c8a17375a54b06f65da10c7"}
Aug 05 17:13:34.466376 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.466345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq" event={"ID":"4effde24-1170-4722-97ff-dda58d6a09b7","Type":"ContainerStarted","Data":"acbafcf9623438cae283f8cd97cfceda4140a1f4b6f077997a0220aa29437f11"}
Aug 05 17:13:34.467904 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.467882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv" event={"ID":"0442c145-eda8-402a-9e11-4ee7c7f1237c","Type":"ContainerStarted","Data":"9fa3ed4164a0bea01e42d23f59d756768dcbc394dc13ef6f664ce1530e53ff4f"}
Aug 05 17:13:34.469261 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.469235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-452n4" event={"ID":"bc5be61d-7ca6-4860-ac6d-92cf4d56ac51","Type":"ContainerStarted","Data":"907d6a410965fcf4ef08371d7a1bf519f124e4738c2578fa2d4bb3f29760f7d8"}
Aug 05 17:13:34.470930 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.470912    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zf7p6" event={"ID":"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d","Type":"ContainerStarted","Data":"1397c3ad97054400c7ade7221acf7b697f0038a7baf977c27e0a9fa59b574eb3"}
Aug 05 17:13:34.471025 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.470944    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zf7p6" event={"ID":"97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d","Type":"ContainerStarted","Data":"a0fba28d1b51eb1301a6d9dbe73a35fc7567060d8b19d1f27f1e19d1bcdd5f95"}
Aug 05 17:13:34.471079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.471050    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:34.516991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.516955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:34.534035 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.533991    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zf7p6" podStartSLOduration=33.649847105 podStartE2EDuration="38.533975046s" totalImagesPullingTime="4.884127941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:34.533644737 +0000 UTC m=+73.066828983" watchObservedRunningTime="2026-08-05 17:13:34.533975046 +0000 UTC m=+73.067159292"
Aug 05 17:13:34.534638 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.534612    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9ftmg" podStartSLOduration=60.684963144 podStartE2EDuration="1m5.534604261s" totalImagesPullingTime="4.849641117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:34.501826435 +0000 UTC m=+73.035010680" watchObservedRunningTime="2026-08-05 17:13:34.534604261 +0000 UTC m=+73.067788505"
Aug 05 17:13:34.556103 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.556034    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-452n4" podStartSLOduration=65.455248754 podStartE2EDuration="1m11.556019492s" totalImagesPullingTime="6.100770738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:34.555552972 +0000 UTC m=+73.088737216" watchObservedRunningTime="2026-08-05 17:13:34.556019492 +0000 UTC m=+73.089203739"
Aug 05 17:13:34.571187 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.571155    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-2mnvv" podStartSLOduration=51.923907037 podStartE2EDuration="56.571144073s" totalImagesPullingTime="4.647237036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:34.570672787 +0000 UTC m=+73.103857055" watchObservedRunningTime="2026-08-05 17:13:34.571144073 +0000 UTC m=+73.104328317"
Aug 05 17:13:34.593447 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.593414    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-ph4zq" podStartSLOduration=60.629750038 podStartE2EDuration="1m5.593406375s" totalImagesPullingTime="4.963656337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:34.592597982 +0000 UTC m=+73.125782423" watchObservedRunningTime="2026-08-05 17:13:34.593406375 +0000 UTC m=+73.126590620"
Aug 05 17:13:34.604477 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.604458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:34.705537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.705503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:34.949125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.949089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:34.979281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:34.979253    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-28ccc"]
Aug 05 17:13:35.014668 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.014633    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-28ccc"]
Aug 05 17:13:35.014792 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.014732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:35.018442 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.018155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jkzfp\""
Aug 05 17:13:35.018442 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.018353    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 05 17:13:35.018442 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.018441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 05 17:13:35.061945 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.061911    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:35.160640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.160609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:35.171755 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.171727    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkvlp\" (UniqueName: \"kubernetes.io/projected/c067cbc4-7c1e-4563-a39a-83e264d6f620-kube-api-access-hkvlp\") pod \"downloads-77c974ffcf-28ccc\" (UID: \"c067cbc4-7c1e-4563-a39a-83e264d6f620\") " pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:35.267999 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.267967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:35.272975 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.272954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkvlp\" (UniqueName: \"kubernetes.io/projected/c067cbc4-7c1e-4563-a39a-83e264d6f620-kube-api-access-hkvlp\") pod \"downloads-77c974ffcf-28ccc\" (UID: \"c067cbc4-7c1e-4563-a39a-83e264d6f620\") " pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:35.288523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.288501    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkvlp\" (UniqueName: \"kubernetes.io/projected/c067cbc4-7c1e-4563-a39a-83e264d6f620-kube-api-access-hkvlp\") pod \"downloads-77c974ffcf-28ccc\" (UID: \"c067cbc4-7c1e-4563-a39a-83e264d6f620\") " pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:35.326430 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.326400    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:35.378837 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.378812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:35.456338 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.456287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:35.484628 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.484491    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-28ccc"]
Aug 05 17:13:35.487063 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:35.487040    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc067cbc4_7c1e_4563_a39a_83e264d6f620.slice/crio-a4475d57256a2ab4ffafc346fb6dacf6331ff56da0d5a8951f63daacff58b45a WatchSource:0}: Error finding container a4475d57256a2ab4ffafc346fb6dacf6331ff56da0d5a8951f63daacff58b45a: Status 404 returned error can't find the container with id a4475d57256a2ab4ffafc346fb6dacf6331ff56da0d5a8951f63daacff58b45a
Aug 05 17:13:35.567797 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.567720    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:35.672668 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:35.672631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:36.241850 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:36.241804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:36.337212 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:36.337176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:36.480636 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:36.480590    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-28ccc" event={"ID":"c067cbc4-7c1e-4563-a39a-83e264d6f620","Type":"ContainerStarted","Data":"a4475d57256a2ab4ffafc346fb6dacf6331ff56da0d5a8951f63daacff58b45a"}
Aug 05 17:13:36.529943 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:36.529860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:36.590723 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:36.590684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:39.442434 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:39.442401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:40.709479 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:40.709450    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:40.796295 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:40.796258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:40.885328 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:40.885278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:42.045795 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:42.045756    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:42.115879 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:42.115844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:42.821282 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:42.821246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:42.965807 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:42.965770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:43.032902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:43.032849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:44.478629 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:44.478188    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zf7p6"
Aug 05 17:13:44.909816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:44.909740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.009907 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.009878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.114131 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.114096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.138639 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.138610    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"]
Aug 05 17:13:45.141565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.141545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.145501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.145349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Aug 05 17:13:45.145501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.145377    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 05 17:13:45.145501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.145354    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tkdm8\""
Aug 05 17:13:45.145501 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.145356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Aug 05 17:13:45.156218 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.156195    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"]
Aug 05 17:13:45.172283 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.172232    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9w6gd"]
Aug 05 17:13:45.174541 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.174522    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.176487 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.176468    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"]
Aug 05 17:13:45.178084 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.178058    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 05 17:13:45.178366 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.178347    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 05 17:13:45.178724 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.178703    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lzm78\""
Aug 05 17:13:45.179057 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.179042    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.179393 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.179151    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 05 17:13:45.182487 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.182399    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nt6dt\""
Aug 05 17:13:45.182658 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.182639    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Aug 05 17:13:45.183450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.183431    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Aug 05 17:13:45.183691 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.183674    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Aug 05 17:13:45.199484 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.199463    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"]
Aug 05 17:13:45.217726 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.217699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.251768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.251735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.251925 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.251776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.251925 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.251819    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snxqs\" (UniqueName: \"kubernetes.io/projected/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-kube-api-access-snxqs\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.251925 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.251851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.321041 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.321013    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.353296 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-sys\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.353480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-wtmp\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.353480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.353480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snxqs\" (UniqueName: \"kubernetes.io/projected/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-kube-api-access-snxqs\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.353480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/01834f4c-8be8-4209-9d6f-949a56d6bd66-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.353688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.353688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lcsp\" (UniqueName: \"kubernetes.io/projected/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-api-access-2lcsp\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.353688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.353688 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:45.353629    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Aug 05 17:13:45.353688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-tls\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.353920 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:45.353697    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-tls podName:520f99ef-7563-4ee5-92f5-90e2b0cbdb05 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:45.853677528 +0000 UTC m=+84.386861762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-tls") pod "openshift-state-metrics-6bb4d7b4dd-zdmxs" (UID: "520f99ef-7563-4ee5-92f5-90e2b0cbdb05") : secret "openshift-state-metrics-tls" not found
Aug 05 17:13:45.353920 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353780    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01834f4c-8be8-4209-9d6f-949a56d6bd66-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.353920 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-root\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.353920 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbp7r\" (UniqueName: \"kubernetes.io/projected/a0aece35-3955-43f3-8aca-681004146d2d-kube-api-access-cbp7r\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.353920 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-textfile\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.354158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-accelerators-collector-config\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.354158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.353977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.354158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.354002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0aece35-3955-43f3-8aca-681004146d2d-metrics-client-ca\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.354158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.354021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.354158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.354040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.354158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.354148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.354766 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.354742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.356761 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.356736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.362956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.362916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snxqs\" (UniqueName: \"kubernetes.io/projected/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-kube-api-access-snxqs\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.423108 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.423035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.455203 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbp7r\" (UniqueName: \"kubernetes.io/projected/a0aece35-3955-43f3-8aca-681004146d2d-kube-api-access-cbp7r\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455373 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-textfile\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-accelerators-collector-config\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0aece35-3955-43f3-8aca-681004146d2d-metrics-client-ca\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.455535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.455535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-sys\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455679 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-wtmp\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455679 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-textfile\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455679 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:45.455595    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Aug 05 17:13:45.455793 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:13:45.455680    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-tls podName:01834f4c-8be8-4209-9d6f-949a56d6bd66 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:45.955658021 +0000 UTC m=+84.488842246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-w5wwp" (UID: "01834f4c-8be8-4209-9d6f-949a56d6bd66") : secret "kube-state-metrics-tls" not found
Aug 05 17:13:45.455793 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-wtmp\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455793 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-sys\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455793 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455793 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/01834f4c-8be8-4209-9d6f-949a56d6bd66-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.455992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455828    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lcsp\" (UniqueName: \"kubernetes.io/projected/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-api-access-2lcsp\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.455992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.455992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-tls\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-accelerators-collector-config\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.455992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.455965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01834f4c-8be8-4209-9d6f-949a56d6bd66-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.456242 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.456017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-root\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.456242 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.456088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a0aece35-3955-43f3-8aca-681004146d2d-root\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.456242 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.456187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/01834f4c-8be8-4209-9d6f-949a56d6bd66-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.456415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.456285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0aece35-3955-43f3-8aca-681004146d2d-metrics-client-ca\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.456718 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.456678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01834f4c-8be8-4209-9d6f-949a56d6bd66-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.456718 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.456692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.458458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.458417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.458670 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.458649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-tls\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.458830 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.458809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0aece35-3955-43f3-8aca-681004146d2d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.474748 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.474713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbp7r\" (UniqueName: \"kubernetes.io/projected/a0aece35-3955-43f3-8aca-681004146d2d-kube-api-access-cbp7r\") pod \"node-exporter-9w6gd\" (UID: \"a0aece35-3955-43f3-8aca-681004146d2d\") " pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.474868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.474852    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lcsp\" (UniqueName: \"kubernetes.io/projected/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-api-access-2lcsp\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.487603 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.487581    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9w6gd"
Aug 05 17:13:45.578272 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.578232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:45.860539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.860499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.863248 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.863220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520f99ef-7563-4ee5-92f5-90e2b0cbdb05-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-zdmxs\" (UID: \"520f99ef-7563-4ee5-92f5-90e2b0cbdb05\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:45.961670 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.961629    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:45.964132 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:45.964099    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01834f4c-8be8-4209-9d6f-949a56d6bd66-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-w5wwp\" (UID: \"01834f4c-8be8-4209-9d6f-949a56d6bd66\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:46.056823 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:46.056786    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"
Aug 05 17:13:46.093351 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:46.093309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"
Aug 05 17:13:46.190570 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:46.190477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:46.316659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:46.316622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:46.396775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:46.396746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:47.280493 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:47.280458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:48.166529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.166499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:48.201289 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.201261    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5774df9dc7-plqwr"]
Aug 05 17:13:48.203983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.203959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.207769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.207228    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Aug 05 17:13:48.207769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.207228    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Aug 05 17:13:48.207769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.207228    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Aug 05 17:13:48.207769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.207612    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Aug 05 17:13:48.208465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.208369    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b52p6k0oloqfd\""
Aug 05 17:13:48.209005 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.208635    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nhx9w\""
Aug 05 17:13:48.209005 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.208819    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Aug 05 17:13:48.226249 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.226222    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5774df9dc7-plqwr"]
Aug 05 17:13:48.261513 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.261485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:48.280273 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280408 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6868\" (UniqueName: \"kubernetes.io/projected/82509f2b-5528-433c-a34e-63955e89c9cf-kube-api-access-w6868\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280408 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-tls\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280408 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82509f2b-5528-433c-a34e-63955e89c9cf-metrics-client-ca\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280844 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280844 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-grpc-tls\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280844 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.280844 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.280632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381400 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6868\" (UniqueName: \"kubernetes.io/projected/82509f2b-5528-433c-a34e-63955e89c9cf-kube-api-access-w6868\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-tls\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381798 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381598    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82509f2b-5528-433c-a34e-63955e89c9cf-metrics-client-ca\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.381798 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.382017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.381919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-grpc-tls\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.382607 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.382535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82509f2b-5528-433c-a34e-63955e89c9cf-metrics-client-ca\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.384458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.384436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.384854 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.384789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-tls\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.384941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.384893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.385165 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.385140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-grpc-tls\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.385817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.385794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.386605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.386577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82509f2b-5528-433c-a34e-63955e89c9cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.391447 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.391424    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6868\" (UniqueName: \"kubernetes.io/projected/82509f2b-5528-433c-a34e-63955e89c9cf-kube-api-access-w6868\") pod \"thanos-querier-5774df9dc7-plqwr\" (UID: \"82509f2b-5528-433c-a34e-63955e89c9cf\") " pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.516374 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.516341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:13:48.533808 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.533773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:48.665943 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:48.665906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:49.040833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:49.040792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:49.097605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:49.097573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:49.906001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:49.905965    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:49.989328 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:49.989278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:50.231113 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:50.231078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:50.452207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:50.452178    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:13:50.736778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:50.736747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:50.800271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:50.800237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:51.398609 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:51.398568    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0aece35_3955_43f3_8aca_681004146d2d.slice/crio-8e1061310ef48fb86a622a166559653a9a40bc69a1807b52075fc1653e444401 WatchSource:0}: Error finding container 8e1061310ef48fb86a622a166559653a9a40bc69a1807b52075fc1653e444401: Status 404 returned error can't find the container with id 8e1061310ef48fb86a622a166559653a9a40bc69a1807b52075fc1653e444401
Aug 05 17:13:51.516595 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.515761    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:13:51.520472 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.520450    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.525139 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.524711    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 05 17:13:51.525139 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.524882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:51.525139 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.524969    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4q9mx\""
Aug 05 17:13:51.525139 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.525016    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 05 17:13:51.525139 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.525098    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 05 17:13:51.525452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.525431    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 05 17:13:51.525912 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.525894    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 05 17:13:51.526202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.526185    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 05 17:13:51.526246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.526217    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 05 17:13:51.526585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.526568    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 05 17:13:51.526671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.526655    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-edj3pm1qj3d7u\""
Aug 05 17:13:51.527647 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.527481    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 05 17:13:51.527647 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.527535    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 05 17:13:51.527939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.527728    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 05 17:13:51.528768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.528619    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 05 17:13:51.529936 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.529918    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 05 17:13:51.530992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.530804    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 05 17:13:51.540364 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.537777    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 05 17:13:51.551631 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.551608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:13:51.555605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.555577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w6gd" event={"ID":"a0aece35-3955-43f3-8aca-681004146d2d","Type":"ContainerStarted","Data":"8e1061310ef48fb86a622a166559653a9a40bc69a1807b52075fc1653e444401"}
Aug 05 17:13:51.574633 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.574612    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs"]
Aug 05 17:13:51.577470 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:51.577437    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520f99ef_7563_4ee5_92f5_90e2b0cbdb05.slice/crio-2063ba25c8ba07e6f0c4d00d287234573aec4b869de3e8bf55d5573066f643fa WatchSource:0}: Error finding container 2063ba25c8ba07e6f0c4d00d287234573aec4b869de3e8bf55d5573066f643fa: Status 404 returned error can't find the container with id 2063ba25c8ba07e6f0c4d00d287234573aec4b869de3e8bf55d5573066f643fa
Aug 05 17:13:51.607995 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.607969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608088 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608088 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608161 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608161 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-config-out\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608227 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608227 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608287 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608287 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608250    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-web-config\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608365 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608365 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608365 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-config\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5nrj\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-kube-api-access-x5nrj\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.608662 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.608559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-config\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709623 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709623 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5nrj\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-kube-api-access-x5nrj\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709724 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709724 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709724 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.709868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-config-out\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.709998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.710032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.710054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-web-config\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.710082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.710108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.710150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.710149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.712331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.711143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.712331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.711728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.712331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.711987    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.712331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.711737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.714097 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.714074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.714412 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.714389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.714708 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.714685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.715112 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.715087    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-config\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.715198 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.715123    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.715448 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.715429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.717765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.717717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.719382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.718394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.719382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.718796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-web-config\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.719382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.718836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.719382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.718983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-config-out\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.719382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.719339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.720222 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.720184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.720222 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.720204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.720628 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.720610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.723670 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.723628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5nrj\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-kube-api-access-x5nrj\") pod \"prometheus-k8s-0\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:51.787241 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.787204    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp"]
Aug 05 17:13:51.788383 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.788354    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5774df9dc7-plqwr"]
Aug 05 17:13:51.792349 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:51.792293    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82509f2b_5528_433c_a34e_63955e89c9cf.slice/crio-0304b27df31b5e17035d8da1e2b14e5edad4bebd8ba588a9b7346392ef68cbb4 WatchSource:0}: Error finding container 0304b27df31b5e17035d8da1e2b14e5edad4bebd8ba588a9b7346392ef68cbb4: Status 404 returned error can't find the container with id 0304b27df31b5e17035d8da1e2b14e5edad4bebd8ba588a9b7346392ef68cbb4
Aug 05 17:13:51.792896 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:51.792867    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01834f4c_8be8_4209_9d6f_949a56d6bd66.slice/crio-fc1898131e8e5dc5ee8eaea8364ffd8425acd01560661b45c8629dd22c85e72a WatchSource:0}: Error finding container fc1898131e8e5dc5ee8eaea8364ffd8425acd01560661b45c8629dd22c85e72a: Status 404 returned error can't find the container with id fc1898131e8e5dc5ee8eaea8364ffd8425acd01560661b45c8629dd22c85e72a
Aug 05 17:13:51.845565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:51.845532    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:13:52.023503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.023396    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:13:52.028536 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:13:52.028499    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf093289f_7242_4ca1_ac81_94b082839da4.slice/crio-10aa27b6bdd5dbf32b11062c8047fcc19dafe24deb191d712baea9ff3206d714 WatchSource:0}: Error finding container 10aa27b6bdd5dbf32b11062c8047fcc19dafe24deb191d712baea9ff3206d714: Status 404 returned error can't find the container with id 10aa27b6bdd5dbf32b11062c8047fcc19dafe24deb191d712baea9ff3206d714
Aug 05 17:13:52.463579 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.462444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:52.564908 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.564367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"10aa27b6bdd5dbf32b11062c8047fcc19dafe24deb191d712baea9ff3206d714"}
Aug 05 17:13:52.567049 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.566072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp" event={"ID":"01834f4c-8be8-4209-9d6f-949a56d6bd66","Type":"ContainerStarted","Data":"fc1898131e8e5dc5ee8eaea8364ffd8425acd01560661b45c8629dd22c85e72a"}
Aug 05 17:13:52.569017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.568990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-28ccc" event={"ID":"c067cbc4-7c1e-4563-a39a-83e264d6f620","Type":"ContainerStarted","Data":"8070c4fd5aa85c4dbc689c374bfbc2e5e13136193c7145e78ee16e0203ca2841"}
Aug 05 17:13:52.569847 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.569826    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:52.573714 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.573688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs" event={"ID":"520f99ef-7563-4ee5-92f5-90e2b0cbdb05","Type":"ContainerStarted","Data":"f7375ef79730b90084e7c32910e3dcf584fcf5382be448cc3d9eb7d7953bb5dc"}
Aug 05 17:13:52.573810 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.573743    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs" event={"ID":"520f99ef-7563-4ee5-92f5-90e2b0cbdb05","Type":"ContainerStarted","Data":"3927d256e6dcab1504266f9ad0fd702d84bda1227f0be77ed043a7883dfc9987"}
Aug 05 17:13:52.573810 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.573773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs" event={"ID":"520f99ef-7563-4ee5-92f5-90e2b0cbdb05","Type":"ContainerStarted","Data":"2063ba25c8ba07e6f0c4d00d287234573aec4b869de3e8bf55d5573066f643fa"}
Aug 05 17:13:52.575783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.575702    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"0304b27df31b5e17035d8da1e2b14e5edad4bebd8ba588a9b7346392ef68cbb4"}
Aug 05 17:13:52.584465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.584440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:52.588572 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.588545    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-28ccc"
Aug 05 17:13:52.621753 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:52.621698    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-28ccc" podStartSLOduration=2.654301849 podStartE2EDuration="18.621679233s" totalImagesPullingTime="15.967377384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:52.601262959 +0000 UTC m=+91.134447205" watchObservedRunningTime="2026-08-05 17:13:52.621679233 +0000 UTC m=+91.154863479"
Aug 05 17:13:56.284053 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.283882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:56.385738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.385703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:56.597381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.597279    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49" exitCode=0
Aug 05 17:13:56.597381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.597359    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"}
Aug 05 17:13:56.600292 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.600262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp" event={"ID":"01834f4c-8be8-4209-9d6f-949a56d6bd66","Type":"ContainerStarted","Data":"873b606da95f3423aa7429723db29f7fe89fd0662630d9115299a13be0ee5d7a"}
Aug 05 17:13:56.600423 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.600337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp" event={"ID":"01834f4c-8be8-4209-9d6f-949a56d6bd66","Type":"ContainerStarted","Data":"3729e4532529900caa385d8ee75e267fe58072f5ed74c081a7d90828d8012640"}
Aug 05 17:13:56.600423 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.600371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp" event={"ID":"01834f4c-8be8-4209-9d6f-949a56d6bd66","Type":"ContainerStarted","Data":"8c02257968c3241d8fc00bc69538636c93e674df6747e9b5269961d12eb6dbe6"}
Aug 05 17:13:56.602293 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.602263    2562 generic.go:356] "Generic (PLEG): container finished" podID="a0aece35-3955-43f3-8aca-681004146d2d" containerID="5e8ec5fabb1f67147773b590217ab37b4435758e72ca5f1d2c4d1d05af8f8d6d" exitCode=0
Aug 05 17:13:56.602426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.602356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w6gd" event={"ID":"a0aece35-3955-43f3-8aca-681004146d2d","Type":"ContainerDied","Data":"5e8ec5fabb1f67147773b590217ab37b4435758e72ca5f1d2c4d1d05af8f8d6d"}
Aug 05 17:13:56.605283 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.605255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs" event={"ID":"520f99ef-7563-4ee5-92f5-90e2b0cbdb05","Type":"ContainerStarted","Data":"8363bcaf1a7c55a404412c0436e0522f57a0d031f42254c3e69ad80731590b74"}
Aug 05 17:13:56.607858 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.607833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"f457ade40928bd9bbf2fb2be43d7a4c2fd6739663292817be1363604b9f5f6c0"}
Aug 05 17:13:56.607948 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.607891    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"6044281f5855deac09dfa78ce20f84e5d8e53cb434a4a80a80affc4e44ee0b05"}
Aug 05 17:13:56.607948 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.607920    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"f8431c815adb429535e5f608a64735ec55f3f403be3d644e2013df0e49f5472e"}
Aug 05 17:13:56.674941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.674887    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-w5wwp" podStartSLOduration=7.731279108 podStartE2EDuration="11.674866414s" totalImagesPullingTime="3.943587306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:56.67348735 +0000 UTC m=+95.206671606" watchObservedRunningTime="2026-08-05 17:13:56.674866414 +0000 UTC m=+95.208050667"
Aug 05 17:13:56.699354 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.699291    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-zdmxs" podStartSLOduration=7.663171632 podStartE2EDuration="11.699271113s" totalImagesPullingTime="4.036099481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:56.696762841 +0000 UTC m=+95.229947085" watchObservedRunningTime="2026-08-05 17:13:56.699271113 +0000 UTC m=+95.232455359"
Aug 05 17:13:56.866620 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:56.866520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:57.092506 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.092474    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66b94b8485-bqdrx"]
Aug 05 17:13:57.189611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.189532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:57.522519 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.522485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:57.616971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.616933    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w6gd" event={"ID":"a0aece35-3955-43f3-8aca-681004146d2d","Type":"ContainerStarted","Data":"deae4f31532b9db6c48ce8a4f8a34f270a9f0ca12948d084ea1332ff0f778624"}
Aug 05 17:13:57.617126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.617002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9w6gd" event={"ID":"a0aece35-3955-43f3-8aca-681004146d2d","Type":"ContainerStarted","Data":"036afb6ca3cd2aa4406222f0d3e27c3ea46501dbdb813936fd6e9671d8142511"}
Aug 05 17:13:57.622125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.622095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:57.645910 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.645366    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9w6gd" podStartSLOduration=8.170915306 podStartE2EDuration="12.645346173s" totalImagesPullingTime="4.332735242s" totalInitContainerRuntime="141.695625ms" isStatefulPod=true podCreationTimestamp="2026-08-05 17:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:57.643979039 +0000 UTC m=+96.177163285" watchObservedRunningTime="2026-08-05 17:13:57.645346173 +0000 UTC m=+96.178530450"
Aug 05 17:13:57.715122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:57.715002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:13:58.637013 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:58.636811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"bdcb0d172c5f335ddfee00994f5d651a7ab1b889c09fd529d62eb64719dc1ddb"}
Aug 05 17:13:58.637013 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:58.636938    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"37354863b520b9adf777ca20946b02f2f92b078b691c37aa07d7e89f84b56d84"}
Aug 05 17:13:58.637013 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:58.636983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" event={"ID":"82509f2b-5528-433c-a34e-63955e89c9cf","Type":"ContainerStarted","Data":"0a6c5c59986e9f485e0ca906fa32d8a3c7bf3972765d4404223faf597fa09c18"}
Aug 05 17:13:58.668333 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:58.668252    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr" podStartSLOduration=4.891276635 podStartE2EDuration="10.668231162s" totalImagesPullingTime="5.776954527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:58.666369683 +0000 UTC m=+97.199553952" watchObservedRunningTime="2026-08-05 17:13:58.668231162 +0000 UTC m=+97.201415408"
Aug 05 17:13:59.433517 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:59.433479    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fgk52"
Aug 05 17:13:59.643178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:13:59.643141    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:14:00.649192 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:00.649150    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"}
Aug 05 17:14:01.215010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.214984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:01.416079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.416002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:01.657150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.657112    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"}
Aug 05 17:14:01.657604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.657195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"}
Aug 05 17:14:01.657604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.657240    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"}
Aug 05 17:14:01.657604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.657274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"}
Aug 05 17:14:01.657604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.657328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerStarted","Data":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"}
Aug 05 17:14:01.703207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.703154    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.054585504 podStartE2EDuration="10.703136191s" totalImagesPullingTime="7.616354273s" totalInitContainerRuntime="32.196414ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:14:01.699662511 +0000 UTC m=+100.232846756" watchObservedRunningTime="2026-08-05 17:14:01.703136191 +0000 UTC m=+100.236320436"
Aug 05 17:14:01.846207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:01.846171    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:14:04.659564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:04.659533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:04.760871 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:04.760836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:05.665820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:05.665794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5774df9dc7-plqwr"
Aug 05 17:14:06.281544 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:06.281513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:06.381481 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:06.381448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:06.749116 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:06.749081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:06.891428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:06.891393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:07.658762 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:07.658730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:10.270512 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:10.270481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:10.371058 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:10.371023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:12.648252 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:12.648217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:12.749010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:12.748980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:17.727876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:17.727841    2562 generic.go:356] "Generic (PLEG): container finished" podID="df0256bf-e00b-43ec-96ef-ad9e6f2717c9" containerID="76e8bc81a79a1a5d8216885719067fa3b6fb4cf332d34f5e984c1aedd4f9eca1" exitCode=0
Aug 05 17:14:17.728433 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:17.727924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7" event={"ID":"df0256bf-e00b-43ec-96ef-ad9e6f2717c9","Type":"ContainerDied","Data":"76e8bc81a79a1a5d8216885719067fa3b6fb4cf332d34f5e984c1aedd4f9eca1"}
Aug 05 17:14:17.728433 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:17.728305    2562 scope.go:117] "RemoveContainer" containerID="76e8bc81a79a1a5d8216885719067fa3b6fb4cf332d34f5e984c1aedd4f9eca1"
Aug 05 17:14:17.729525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:17.729502    2562 generic.go:356] "Generic (PLEG): container finished" podID="02c0d02c-18e8-456c-a4d0-2af2ad30c7ed" containerID="5f3c0455b3a53324e211ae0966fb95bbf80dbbf8a0ca5bdf39510bca39154e2d" exitCode=0
Aug 05 17:14:17.729645 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:17.729555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767" event={"ID":"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed","Type":"ContainerDied","Data":"5f3c0455b3a53324e211ae0966fb95bbf80dbbf8a0ca5bdf39510bca39154e2d"}
Aug 05 17:14:17.729831 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:17.729817    2562 scope.go:117] "RemoveContainer" containerID="5f3c0455b3a53324e211ae0966fb95bbf80dbbf8a0ca5bdf39510bca39154e2d"
Aug 05 17:14:18.739202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:18.739165    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-q5wr7" event={"ID":"df0256bf-e00b-43ec-96ef-ad9e6f2717c9","Type":"ContainerStarted","Data":"8e03dd1f799e72178972596e70fcad7e4f6946d1ce065984754e2c95f2ccc672"}
Aug 05 17:14:18.740888 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:18.740858    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-gm767" event={"ID":"02c0d02c-18e8-456c-a4d0-2af2ad30c7ed","Type":"ContainerStarted","Data":"e05c9e5750a94fdaf48e2d0e08c9bce7e05d49a5faf533478dc8710b3d3ee17e"}
Aug 05 17:14:20.578090 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:20.578053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:20.938004 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:20.937928    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:22.121988 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.121935    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx" podUID="02bfa378-627d-499b-84c9-eb1c9f19a574" containerName="registry" containerID="cri-o://d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b" gracePeriod=30
Aug 05 17:14:22.367738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.367714    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:14:22.505038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-installation-pull-secrets\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505053    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02bfa378-627d-499b-84c9-eb1c9f19a574-ca-trust-extracted\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505093    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-bound-sa-token\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505120    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505148    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-trusted-ca\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505356    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4sxx\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-kube-api-access-k4sxx\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505431    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-certificates\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505483    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-image-registry-private-configuration\") pod \"02bfa378-627d-499b-84c9-eb1c9f19a574\" (UID: \"02bfa378-627d-499b-84c9-eb1c9f19a574\") "
Aug 05 17:14:22.505753 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505633    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:14:22.505912 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.505883    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:14:22.507641 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.507616    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:14:22.507859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.507774    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:14:22.507859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.507787    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:14:22.507967 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.507928    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-kube-api-access-k4sxx" (OuterVolumeSpecName: "kube-api-access-k4sxx") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "kube-api-access-k4sxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:14:22.508036 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.508008    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:14:22.514289 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.514265    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02bfa378-627d-499b-84c9-eb1c9f19a574-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "02bfa378-627d-499b-84c9-eb1c9f19a574" (UID: "02bfa378-627d-499b-84c9-eb1c9f19a574"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:14:22.607122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607094    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4sxx\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-kube-api-access-k4sxx\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607117    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-certificates\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607128    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-image-registry-private-configuration\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607298 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607138    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/02bfa378-627d-499b-84c9-eb1c9f19a574-installation-pull-secrets\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607298 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607150    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/02bfa378-627d-499b-84c9-eb1c9f19a574-ca-trust-extracted\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607298 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607160    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-bound-sa-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607298 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607168    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/02bfa378-627d-499b-84c9-eb1c9f19a574-registry-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.607298 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.607176    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa378-627d-499b-84c9-eb1c9f19a574-trusted-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:14:22.757402 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.757297    2562 generic.go:356] "Generic (PLEG): container finished" podID="02bfa378-627d-499b-84c9-eb1c9f19a574" containerID="d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b" exitCode=0
Aug 05 17:14:22.757402 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.757358    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx"
Aug 05 17:14:22.757402 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.757383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx" event={"ID":"02bfa378-627d-499b-84c9-eb1c9f19a574","Type":"ContainerDied","Data":"d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b"}
Aug 05 17:14:22.757656 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.757447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b94b8485-bqdrx" event={"ID":"02bfa378-627d-499b-84c9-eb1c9f19a574","Type":"ContainerDied","Data":"b2243266ce0212b36799d169c1bcc7c5493ec8b7ae7e80b3f3a7184d92938425"}
Aug 05 17:14:22.757656 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.757474    2562 scope.go:117] "RemoveContainer" containerID="d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b"
Aug 05 17:14:22.766416 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.766396    2562 scope.go:117] "RemoveContainer" containerID="d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b"
Aug 05 17:14:22.766676 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:14:22.766649    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b\": container with ID starting with d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b not found: ID does not exist" containerID="d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b"
Aug 05 17:14:22.766752 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.766682    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b"} err="failed to get container status \"d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b\": rpc error: code = NotFound desc = could not find container \"d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b\": container with ID starting with d6749fa13a9a5af2c94b3f97e5059f7cb403f5dd2fba43a2adf55e8140a6484b not found: ID does not exist"
Aug 05 17:14:22.788451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.788426    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66b94b8485-bqdrx"]
Aug 05 17:14:22.791295 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:22.791270    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66b94b8485-bqdrx"]
Aug 05 17:14:23.032750 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:23.032683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:23.135941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:23.135909    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:23.200757 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:23.200724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:24.101704 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:24.101673    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bfa378-627d-499b-84c9-eb1c9f19a574" path="/var/lib/kubelet/pods/02bfa378-627d-499b-84c9-eb1c9f19a574/volumes"
Aug 05 17:14:27.780619 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:27.780583    2562 generic.go:356] "Generic (PLEG): container finished" podID="8f9aa898-2769-4d57-9eac-c585ad07a01a" containerID="da3eb6b37b4b09548ef7bd4f30e3d5b5cae1c3bf23bb9860a0c8f8a23a4aa127" exitCode=0
Aug 05 17:14:27.781172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:27.780656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-97hsq" event={"ID":"8f9aa898-2769-4d57-9eac-c585ad07a01a","Type":"ContainerDied","Data":"da3eb6b37b4b09548ef7bd4f30e3d5b5cae1c3bf23bb9860a0c8f8a23a4aa127"}
Aug 05 17:14:27.781172 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:27.780976    2562 scope.go:117] "RemoveContainer" containerID="da3eb6b37b4b09548ef7bd4f30e3d5b5cae1c3bf23bb9860a0c8f8a23a4aa127"
Aug 05 17:14:28.787387 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:28.787354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-97hsq" event={"ID":"8f9aa898-2769-4d57-9eac-c585ad07a01a","Type":"ContainerStarted","Data":"fc90fce7b2817dd744ada62f2357959ed8dec80f83d47ec93374573838acbb1b"}
Aug 05 17:14:35.259974 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:35.259901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:35.323502 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:35.323476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:49.198851 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:49.198815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:14:51.845971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:51.845929    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:14:51.861816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:51.861788    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:14:51.904543 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:14:51.904515    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:04.572214 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:04.572181    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:15:04.672096 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:04.672058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:15:10.111282 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111251    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:15:10.111820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111538    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="prometheus" containerID="cri-o://a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc" gracePeriod=600
Aug 05 17:15:10.111820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111566    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-web" containerID="cri-o://9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343" gracePeriod=600
Aug 05 17:15:10.111820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111561    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce" gracePeriod=600
Aug 05 17:15:10.111820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111586    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="thanos-sidecar" containerID="cri-o://2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c" gracePeriod=600
Aug 05 17:15:10.111820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111589    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy" containerID="cri-o://80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51" gracePeriod=600
Aug 05 17:15:10.111820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.111614    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="config-reloader" containerID="cri-o://9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903" gracePeriod=600
Aug 05 17:15:10.357235 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.357209    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:10.515355 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515300    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-config-out\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515355 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515360    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-grpc-tls\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515391    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515412    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-web-config\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515428    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-kubelet-serving-ca-bundle\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515451    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515472    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-config\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515503    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-db\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515541    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-metrics-client-ca\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.515589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515565    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-0\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515595    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-metrics-client-certs\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-serving-certs-ca-bundle\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515650    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-tls\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515681    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5nrj\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-kube-api-access-x5nrj\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515715    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-trusted-ca-bundle\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515789    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-kube-rbac-proxy\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515818    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-thanos-prometheus-http-client-file\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-tls-assets\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515890    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-1\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515925    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-2\") pod \"f093289f-7242-4ca1-ac81-94b082839da4\" (UID: \"f093289f-7242-4ca1-ac81-94b082839da4\") "
Aug 05 17:15:10.516001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.515932    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.516580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.516174    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.516580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.516531    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.517445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.517145    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:15:10.517445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.517280    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.517885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.517860    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.518123 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.518233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518159    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.518682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518404    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-config-out" (OuterVolumeSpecName: "config-out") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:15:10.518682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.518831 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518759    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.518831 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.519060 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.518841    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.519188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.519122    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:10.519661 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.519636    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-config" (OuterVolumeSpecName: "config") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.520173 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.520154    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.520772 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.520757    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.520910 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.520891    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:15:10.520967 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.520890    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.521071 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.521055    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-kube-api-access-x5nrj" (OuterVolumeSpecName: "kube-api-access-x5nrj") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "kube-api-access-x5nrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:15:10.529651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.529628    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-web-config" (OuterVolumeSpecName: "web-config") pod "f093289f-7242-4ca1-ac81-94b082839da4" (UID: "f093289f-7242-4ca1-ac81-94b082839da4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:10.616729 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616681    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-config-out\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616729 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616722    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-grpc-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616729 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616734    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616729 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616744    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-web-config\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616753    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616761    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-config\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616769    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-db\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616779    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-metrics-client-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616789    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616797    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-metrics-client-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616806    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616814    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616822    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5nrj\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-kube-api-access-x5nrj\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616830    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616840    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-secret-kube-rbac-proxy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616848    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f093289f-7242-4ca1-ac81-94b082839da4-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616859    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f093289f-7242-4ca1-ac81-94b082839da4-tls-assets\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616867    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.616979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.616875    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f093289f-7242-4ca1-ac81-94b082839da4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:15:10.960923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960888    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce" exitCode=0
Aug 05 17:15:10.960923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960915    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51" exitCode=0
Aug 05 17:15:10.960923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960922    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343" exitCode=0
Aug 05 17:15:10.960923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960928    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c" exitCode=0
Aug 05 17:15:10.960923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960933    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903" exitCode=0
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960939    2562 generic.go:356] "Generic (PLEG): container finished" podID="f093289f-7242-4ca1-ac81-94b082839da4" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc" exitCode=0
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960941    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"}
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.960992    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961042    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"}
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"}
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961095    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"}
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961107    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"}
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"}
Aug 05 17:15:10.961330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.961184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f093289f-7242-4ca1-ac81-94b082839da4","Type":"ContainerDied","Data":"10aa27b6bdd5dbf32b11062c8047fcc19dafe24deb191d712baea9ff3206d714"}
Aug 05 17:15:10.969704 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.969684    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:10.977654 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.977638    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:10.984435 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.984418    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:10.991413 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.991389    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:10.992848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.992825    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:15:10.998879 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.998856    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:15:10.998974 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:10.998857    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.005955 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.005939    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.012861 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.012840    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.017466 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.017441    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.017537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.017470    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"} err="failed to get container status \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist"
Aug 05 17:15:11.017537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.017489    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.017728 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.017708    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.017788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.017737    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"} err="failed to get container status \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist"
Aug 05 17:15:11.017788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.017759    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.018034 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.018018    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.018072 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018039    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"} err="failed to get container status \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist"
Aug 05 17:15:11.018072 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018054    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.018273 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.018248    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.018325 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018278    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"} err="failed to get container status \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist"
Aug 05 17:15:11.018325 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018292    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.018577 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.018561    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.018648 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018583    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"} err="failed to get container status \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist"
Aug 05 17:15:11.018648 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018602    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.018851 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.018833    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.018903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018856    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"} err="failed to get container status \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist"
Aug 05 17:15:11.018903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.018869    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.019085 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:15:11.019067    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.019146 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019092    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"} err="failed to get container status \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist"
Aug 05 17:15:11.019146 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019112    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.019338 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019305    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"} err="failed to get container status \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist"
Aug 05 17:15:11.019383 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019339    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.019548 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019529    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"} err="failed to get container status \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist"
Aug 05 17:15:11.019590 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019549    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.019787 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019770    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"} err="failed to get container status \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist"
Aug 05 17:15:11.019829 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019788    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.019990 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019975    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"} err="failed to get container status \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist"
Aug 05 17:15:11.020040 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.019990    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.020211 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020193    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"} err="failed to get container status \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist"
Aug 05 17:15:11.020258 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020211    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.020451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020432    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"} err="failed to get container status \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist"
Aug 05 17:15:11.020523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020453    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.020666 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020645    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"} err="failed to get container status \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist"
Aug 05 17:15:11.020750 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020667    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.020892 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020872    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"} err="failed to get container status \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist"
Aug 05 17:15:11.020957 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.020893    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.021079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021063    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"} err="failed to get container status \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist"
Aug 05 17:15:11.021124 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021080    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.021291 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021276    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"} err="failed to get container status \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist"
Aug 05 17:15:11.021357 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021292    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.021628 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021607    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"} err="failed to get container status \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist"
Aug 05 17:15:11.021628 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021628    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.021817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021802    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"} err="failed to get container status \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist"
Aug 05 17:15:11.021817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021816    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.021995 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021976    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"} err="failed to get container status \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist"
Aug 05 17:15:11.022044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.021995    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.022192 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022177    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"} err="failed to get container status \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist"
Aug 05 17:15:11.022237 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022192    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.022410 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022393    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"} err="failed to get container status \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist"
Aug 05 17:15:11.022452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022410    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.022611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022597    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"} err="failed to get container status \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist"
Aug 05 17:15:11.022651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022611    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.022777 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022761    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"} err="failed to get container status \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist"
Aug 05 17:15:11.022827 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022777    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.022970 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022954    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"} err="failed to get container status \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist"
Aug 05 17:15:11.023011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.022971    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.023185 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023166    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"} err="failed to get container status \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist"
Aug 05 17:15:11.023232 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023186    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.023376 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023361    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"} err="failed to get container status \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist"
Aug 05 17:15:11.023415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023376    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.023520 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023505    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"} err="failed to get container status \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist"
Aug 05 17:15:11.023573 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023521    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.023722 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023703    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"} err="failed to get container status \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist"
Aug 05 17:15:11.023795 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023724    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.023935 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023917    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"} err="failed to get container status \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist"
Aug 05 17:15:11.023982 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.023936    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.024138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.024122    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"} err="failed to get container status \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist"
Aug 05 17:15:11.024195 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.024138    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.024365 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.024344    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"} err="failed to get container status \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist"
Aug 05 17:15:11.024365 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.024365    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.025422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.025406    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"} err="failed to get container status \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist"
Aug 05 17:15:11.025471 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.025423    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.025622 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.025603    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"} err="failed to get container status \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist"
Aug 05 17:15:11.025684 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.025622    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.025802 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.025788    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"} err="failed to get container status \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist"
Aug 05 17:15:11.025843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.025804    2562 scope.go:117] "RemoveContainer" containerID="8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"
Aug 05 17:15:11.026058 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.026028    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce"} err="failed to get container status \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": rpc error: code = NotFound desc = could not find container \"8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce\": container with ID starting with 8556e119bab0b613174799b4584977afe5cd73c41f8718333f827abab2ade1ce not found: ID does not exist"
Aug 05 17:15:11.026058 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.026050    2562 scope.go:117] "RemoveContainer" containerID="80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"
Aug 05 17:15:11.026479 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.026393    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51"} err="failed to get container status \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": rpc error: code = NotFound desc = could not find container \"80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51\": container with ID starting with 80b1ae2c3a807c3dab4e8a2277d90df8b5372d255309ff078255e0f39ec9db51 not found: ID does not exist"
Aug 05 17:15:11.026552 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.026502    2562 scope.go:117] "RemoveContainer" containerID="9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"
Aug 05 17:15:11.026939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.026921    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343"} err="failed to get container status \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": rpc error: code = NotFound desc = could not find container \"9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343\": container with ID starting with 9ba65762cfc436200cd84b03780461edbb684e235f0dff645647c465fd480343 not found: ID does not exist"
Aug 05 17:15:11.027013 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.026939    2562 scope.go:117] "RemoveContainer" containerID="2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"
Aug 05 17:15:11.027191 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.027169    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c"} err="failed to get container status \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": rpc error: code = NotFound desc = could not find container \"2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c\": container with ID starting with 2f76d870d003b1325cae46451c7ba8f1da5ce1af2414e64a90e693064d848e7c not found: ID does not exist"
Aug 05 17:15:11.027281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.027193    2562 scope.go:117] "RemoveContainer" containerID="9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"
Aug 05 17:15:11.027511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.027478    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903"} err="failed to get container status \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": rpc error: code = NotFound desc = could not find container \"9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903\": container with ID starting with 9b7480042c7b2624f807ff4386e1cef7234ccb1f1d6e701196ab6c7470e77903 not found: ID does not exist"
Aug 05 17:15:11.027511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.027506    2562 scope.go:117] "RemoveContainer" containerID="a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"
Aug 05 17:15:11.027841 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.027819    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc"} err="failed to get container status \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": rpc error: code = NotFound desc = could not find container \"a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc\": container with ID starting with a1f84ea45d5db9f0cf283c23283612bd5a22ff9fec9232952082d19e2497c9cc not found: ID does not exist"
Aug 05 17:15:11.027841 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.027841    2562 scope.go:117] "RemoveContainer" containerID="38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"
Aug 05 17:15:11.028082 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.028064    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49"} err="failed to get container status \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": rpc error: code = NotFound desc = could not find container \"38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49\": container with ID starting with 38705b37caf8f9eabdf4ac654b2239aaf70a2059c5e79fff0489f692cbdabf49 not found: ID does not exist"
Aug 05 17:15:11.044031 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044009    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:15:11.044368 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044354    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044370    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044383    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-thanos"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044389    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-thanos"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044400    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="thanos-sidecar"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044406    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="thanos-sidecar"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044417    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="init-config-reloader"
Aug 05 17:15:11.044422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044422    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="init-config-reloader"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044434    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="prometheus"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044439    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="prometheus"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044446    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="config-reloader"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044450    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="config-reloader"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044456    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bfa378-627d-499b-84c9-eb1c9f19a574" containerName="registry"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044462    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bfa378-627d-499b-84c9-eb1c9f19a574" containerName="registry"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044469    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-web"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044474    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-web"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044522    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="config-reloader"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044529    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bfa378-627d-499b-84c9-eb1c9f19a574" containerName="registry"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044534    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-web"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044541    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="prometheus"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044548    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044554    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="kube-rbac-proxy-thanos"
Aug 05 17:15:11.044674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.044561    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f093289f-7242-4ca1-ac81-94b082839da4" containerName="thanos-sidecar"
Aug 05 17:15:11.048543 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.048527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.051955 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.051934    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 05 17:15:11.052035 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.051985    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 05 17:15:11.057440 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.057425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 05 17:15:11.058269 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.058250    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 05 17:15:11.058506 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.058491    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 05 17:15:11.058556 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.058505    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 05 17:15:11.058631 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.058617    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 05 17:15:11.059838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.059818    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 05 17:15:11.059930 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.059846    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 05 17:15:11.059977 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.059958    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-edj3pm1qj3d7u\""
Aug 05 17:15:11.060066 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.060050    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 05 17:15:11.060132 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.060062    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4q9mx\""
Aug 05 17:15:11.060132 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.060082    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 05 17:15:11.060406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.060392    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 05 17:15:11.061996 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.061979    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 05 17:15:11.062049 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.062035    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 05 17:15:11.064609 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.064593    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 05 17:15:11.071152 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.071131    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:15:11.121524 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-config-out\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121625    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121670    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121723    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.121876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-config\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-web-config\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121948    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.121979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp6hl\" (UniqueName: \"kubernetes.io/projected/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-kube-api-access-jp6hl\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.122012    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.122284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.122045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223244 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223244 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223244 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-config\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-web-config\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223286    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223355    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp6hl\" (UniqueName: \"kubernetes.io/projected/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-kube-api-access-jp6hl\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223381    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-config-out\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223549 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.223931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.223895    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.224443 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.224136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.226611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.226446    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.226611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.226493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.226611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.226527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-web-config\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.226611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.226530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.226939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.226626    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-config\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.226939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.226845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-config-out\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.227126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.227094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.227459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.227265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.227459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.227351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.227655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.227472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.227655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.227560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.227904 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.227885    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.228404 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.228379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.229616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.229596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.229705 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.229603    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.229848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.229829    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.230188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.230168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.233805 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.233787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp6hl\" (UniqueName: \"kubernetes.io/projected/71a3cd67-3ca6-4e4e-9e7f-9150fc195961-kube-api-access-jp6hl\") pod \"prometheus-k8s-0\" (UID: \"71a3cd67-3ca6-4e4e-9e7f-9150fc195961\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.357182 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.357139    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:11.513546 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.513473    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 05 17:15:11.516750 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:15:11.516723    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a3cd67_3ca6_4e4e_9e7f_9150fc195961.slice/crio-a3bf62a4c1b11b46d1b225e171e604e64109783f04507c30a1211b1ba28d27bd WatchSource:0}: Error finding container a3bf62a4c1b11b46d1b225e171e604e64109783f04507c30a1211b1ba28d27bd: Status 404 returned error can't find the container with id a3bf62a4c1b11b46d1b225e171e604e64109783f04507c30a1211b1ba28d27bd
Aug 05 17:15:11.967424 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.967392    2562 generic.go:356] "Generic (PLEG): container finished" podID="71a3cd67-3ca6-4e4e-9e7f-9150fc195961" containerID="c9a2abcc10475b9cd0171ac63bf79afe4576db1bb99ac1bdb80cb36c705e4521" exitCode=0
Aug 05 17:15:11.967603 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.967476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerDied","Data":"c9a2abcc10475b9cd0171ac63bf79afe4576db1bb99ac1bdb80cb36c705e4521"}
Aug 05 17:15:11.967603 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:11.967542    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"a3bf62a4c1b11b46d1b225e171e604e64109783f04507c30a1211b1ba28d27bd"}
Aug 05 17:15:12.103170 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.103140    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f093289f-7242-4ca1-ac81-94b082839da4" path="/var/lib/kubelet/pods/f093289f-7242-4ca1-ac81-94b082839da4/volumes"
Aug 05 17:15:12.769439 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.769408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:15:12.910458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.910423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:15:12.974956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.974925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"5ec09dee0bedf551cacf15d85769ab75b379a9cd66230b40967ffbf7efbf3650"}
Aug 05 17:15:12.975107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.975009    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"79a850a1283d9f4c01e20eb818c531d657e7c31dfc93ef42d188c70c5438a1ea"}
Aug 05 17:15:12.975107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.975036    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"9f64a315f3dbbb0a1fba253d2a14ae09753c710e874051af59484bff82f9a855"}
Aug 05 17:15:12.975107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.975059    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"7e57b71804ee862dc9817e3eee3de0d43029ad5d77a09fb76b56252960a847a3"}
Aug 05 17:15:12.975107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.975090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"9b243cb761541d3f8dd81e461e16b34f4472510364c04c74b39660d9f06c56cd"}
Aug 05 17:15:12.975255 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:12.975114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71a3cd67-3ca6-4e4e-9e7f-9150fc195961","Type":"ContainerStarted","Data":"90425e3a24c9acbe6bf637a33924b27f1c27b6472caf9cb15fec4d19990f7e23"}
Aug 05 17:15:13.018878 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:13.018820    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.999407519 podStartE2EDuration="2.018800477s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.392958ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:15:13.016631583 +0000 UTC m=+171.549815850" watchObservedRunningTime="2026-08-05 17:15:13.018800477 +0000 UTC m=+171.551984723"
Aug 05 17:15:16.357840 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:16.357756    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:15:23.664275 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:23.664245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:15:23.761521 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:23.761467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:15:23.829427 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:15:23.829388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:11.357537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:11.357501    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:16:11.372759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:11.372738    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:16:12.218219 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:12.218186    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 05 17:16:19.985160 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:19.985121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:20.056597 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:20.056565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:29.533409 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:29.533375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:29.594555 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:29.594523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:32.097094 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:32.097037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:32.192882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:32.192848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:35.610484 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:35.610444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:35.733854 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:35.733822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:35.909469 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:35.909388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:40.930755 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:40.930721    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:41.030826 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:41.030795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:16:41.130887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:16:41.130858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:17:21.836690 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:21.836664    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:17:21.839490 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:21.839460    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:17:21.842062 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:21.842042    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:17:21.845153 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:21.845134    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:17:21.849407 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:21.849390    2562 kubelet.go:1648] "Image garbage collection succeeded"
Aug 05 17:17:28.548511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:28.548476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:17:35.482934 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:35.482880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:17:35.578615 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:35.578585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:17:35.644954 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:17:35.644925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:18:36.563241 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:18:36.563205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:12.404887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:12.404852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:12.503217 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:12.503188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:25.444589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:25.444557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:25.722860 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:25.722785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:25.914063 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:25.914031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:26.131918 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:26.131840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:26.513587 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:26.513560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:26.587943 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:26.587915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:26.682121 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:26.682094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:26.920460 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:26.920381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:27.135496 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:27.135471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:27.236382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:27.236358    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:19:27.334896 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:19:27.334870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:13.978744 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:13.978711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:18.931275 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:18.931241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:19.021154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:19.021126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:30.245590 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:30.245563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:30.481278 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:30.481240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:33.876374 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:33.876341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:33.977077 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:33.977048    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:36.535593 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:36.535548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:36.634915 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:36.634887    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:36.738859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:36.738834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:38.409372 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.409334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:38.441104 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.441072    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"]
Aug 05 17:20:38.443346 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.443332    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.446120 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.446082    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 05 17:20:38.446120 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.446114    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 05 17:20:38.447206 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.447186    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-tfgrb\""
Aug 05 17:20:38.453201 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.453183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"]
Aug 05 17:20:38.559128 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.559097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqgkq\" (UniqueName: \"kubernetes.io/projected/8880bd40-adea-48dc-8ed5-5456eef0eb29-kube-api-access-kqgkq\") pod \"cert-manager-cainjector-559cdd8c87-twtkv\" (UID: \"8880bd40-adea-48dc-8ed5-5456eef0eb29\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.559280 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.559156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8880bd40-adea-48dc-8ed5-5456eef0eb29-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-twtkv\" (UID: \"8880bd40-adea-48dc-8ed5-5456eef0eb29\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.655738 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.655703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:38.659889 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.659838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqgkq\" (UniqueName: \"kubernetes.io/projected/8880bd40-adea-48dc-8ed5-5456eef0eb29-kube-api-access-kqgkq\") pod \"cert-manager-cainjector-559cdd8c87-twtkv\" (UID: \"8880bd40-adea-48dc-8ed5-5456eef0eb29\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.659978 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.659890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8880bd40-adea-48dc-8ed5-5456eef0eb29-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-twtkv\" (UID: \"8880bd40-adea-48dc-8ed5-5456eef0eb29\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.677967 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.677937    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqgkq\" (UniqueName: \"kubernetes.io/projected/8880bd40-adea-48dc-8ed5-5456eef0eb29-kube-api-access-kqgkq\") pod \"cert-manager-cainjector-559cdd8c87-twtkv\" (UID: \"8880bd40-adea-48dc-8ed5-5456eef0eb29\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.679523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.679507    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8880bd40-adea-48dc-8ed5-5456eef0eb29-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-twtkv\" (UID: \"8880bd40-adea-48dc-8ed5-5456eef0eb29\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.757609 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.757582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:38.772920 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.772899    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"
Aug 05 17:20:38.899544 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.899517    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-twtkv"]
Aug 05 17:20:38.903025 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:20:38.903002    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8880bd40_adea_48dc_8ed5_5456eef0eb29.slice/crio-839d592c8ae47420851ef00967a94ea448b88bc4d9e6ef2c8eb2eb4684c11326 WatchSource:0}: Error finding container 839d592c8ae47420851ef00967a94ea448b88bc4d9e6ef2c8eb2eb4684c11326: Status 404 returned error can't find the container with id 839d592c8ae47420851ef00967a94ea448b88bc4d9e6ef2c8eb2eb4684c11326
Aug 05 17:20:38.904946 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.904925    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:20:38.985994 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:38.985960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:39.262372 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:39.262277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv" event={"ID":"8880bd40-adea-48dc-8ed5-5456eef0eb29","Type":"ContainerStarted","Data":"839d592c8ae47420851ef00967a94ea448b88bc4d9e6ef2c8eb2eb4684c11326"}
Aug 05 17:20:40.307659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:40.307629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:40.403160 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:40.403103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:40.502664 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:40.502624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:42.280763 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:42.280728    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv" event={"ID":"8880bd40-adea-48dc-8ed5-5456eef0eb29","Type":"ContainerStarted","Data":"6ed5b16056d568ac35b8d60c5d260ac8e4f3d4379885e86734a7ad2272842dc3"}
Aug 05 17:20:42.298388 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:42.298345    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-twtkv" podStartSLOduration=1.089684678 podStartE2EDuration="4.298329792s" totalImagesPullingTime="3.208645114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:42.296639943 +0000 UTC m=+500.829824187" watchObservedRunningTime="2026-08-05 17:20:42.298329792 +0000 UTC m=+500.831514037"
Aug 05 17:20:43.319108 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:43.319077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:43.390390 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:43.390361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:47.782449 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:47.782416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:51.148375 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:51.148344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:51.248939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:51.248908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:20:52.558762 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.558730    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"]
Aug 05 17:20:52.563340 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.563073    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.566592 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.566568    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ks627\""
Aug 05 17:20:52.566592 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.566580    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 05 17:20:52.567720 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.567701    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:20:52.572004 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.571985    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"]
Aug 05 17:20:52.672858 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.672829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g94h7\" (UniqueName: \"kubernetes.io/projected/cb58a5c9-6746-416a-985b-1100145b716a-kube-api-access-g94h7\") pod \"openshift-lws-operator-fd8ccff4c-nk6rm\" (UID: \"cb58a5c9-6746-416a-985b-1100145b716a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.672991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.672869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb58a5c9-6746-416a-985b-1100145b716a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nk6rm\" (UID: \"cb58a5c9-6746-416a-985b-1100145b716a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.773691 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.773667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g94h7\" (UniqueName: \"kubernetes.io/projected/cb58a5c9-6746-416a-985b-1100145b716a-kube-api-access-g94h7\") pod \"openshift-lws-operator-fd8ccff4c-nk6rm\" (UID: \"cb58a5c9-6746-416a-985b-1100145b716a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.773831 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.773704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb58a5c9-6746-416a-985b-1100145b716a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nk6rm\" (UID: \"cb58a5c9-6746-416a-985b-1100145b716a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.774016 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.774001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb58a5c9-6746-416a-985b-1100145b716a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nk6rm\" (UID: \"cb58a5c9-6746-416a-985b-1100145b716a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.782277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.782257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g94h7\" (UniqueName: \"kubernetes.io/projected/cb58a5c9-6746-416a-985b-1100145b716a-kube-api-access-g94h7\") pod \"openshift-lws-operator-fd8ccff4c-nk6rm\" (UID: \"cb58a5c9-6746-416a-985b-1100145b716a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:52.874355 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:52.874273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"
Aug 05 17:20:53.005671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:53.005000    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm"]
Aug 05 17:20:53.008687 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:20:53.008646    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb58a5c9_6746_416a_985b_1100145b716a.slice/crio-74c4bf0259caf9d0530bddbc07b60e496befd2160a7d7397ca722309443aa4b1 WatchSource:0}: Error finding container 74c4bf0259caf9d0530bddbc07b60e496befd2160a7d7397ca722309443aa4b1: Status 404 returned error can't find the container with id 74c4bf0259caf9d0530bddbc07b60e496befd2160a7d7397ca722309443aa4b1
Aug 05 17:20:53.330167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:20:53.330130    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm" event={"ID":"cb58a5c9-6746-416a-985b-1100145b716a","Type":"ContainerStarted","Data":"74c4bf0259caf9d0530bddbc07b60e496befd2160a7d7397ca722309443aa4b1"}
Aug 05 17:21:00.366466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:00.366382    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm" event={"ID":"cb58a5c9-6746-416a-985b-1100145b716a","Type":"ContainerStarted","Data":"d8409fd4f36fe939ea657798b89fe9d7ca08f1d2ca3b48aa69205b0d06510762"}
Aug 05 17:21:00.382816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:00.382769    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nk6rm" podStartSLOduration=1.8357397309999999 podStartE2EDuration="8.382755385s" totalImagesPullingTime="6.547015654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:00.38173935 +0000 UTC m=+518.914923599" watchObservedRunningTime="2026-08-05 17:21:00.382755385 +0000 UTC m=+518.915939631"
Aug 05 17:21:03.762591 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:03.762557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:08.874161 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:08.874129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:08.975908 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:08.975881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:09.041210 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:09.041180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:10.388937 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:10.388900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:14.197024 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.196994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:14.283073 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.283035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:14.284978 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.284954    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"]
Aug 05 17:21:14.288740 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.288721    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.293764 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.293740    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 05 17:21:14.293870 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.293765    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7pxwt\""
Aug 05 17:21:14.294014 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.293992    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 05 17:21:14.294137 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.294035    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 05 17:21:14.301983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.301963    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"]
Aug 05 17:21:14.346428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.346396    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-manager-config\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.346594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.346435    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6f4k\" (UniqueName: \"kubernetes.io/projected/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-kube-api-access-z6f4k\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.346594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.346513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-metrics-cert\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.346594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.346568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-cert\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.447393 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.447336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-manager-config\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.447393 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.447370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6f4k\" (UniqueName: \"kubernetes.io/projected/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-kube-api-access-z6f4k\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.447540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.447488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-metrics-cert\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.447587 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.447546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-cert\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.448017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.447990    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-manager-config\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.449853 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.449830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-metrics-cert\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.449938 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.449897    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-cert\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.458560 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.458537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6f4k\" (UniqueName: \"kubernetes.io/projected/84950fef-5a5e-4abe-9ef5-2fbb89a4f582-kube-api-access-z6f4k\") pod \"lws-controller-manager-77c595c7bb-96fq9\" (UID: \"84950fef-5a5e-4abe-9ef5-2fbb89a4f582\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.598338 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.598283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:14.781726 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:14.781701    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"]
Aug 05 17:21:14.783844 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:21:14.783816    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84950fef_5a5e_4abe_9ef5_2fbb89a4f582.slice/crio-4052cf005515a79863f374fe6ffc330783a00e287917f3d9d4495d44adb09911 WatchSource:0}: Error finding container 4052cf005515a79863f374fe6ffc330783a00e287917f3d9d4495d44adb09911: Status 404 returned error can't find the container with id 4052cf005515a79863f374fe6ffc330783a00e287917f3d9d4495d44adb09911
Aug 05 17:21:15.432710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:15.432676    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9" event={"ID":"84950fef-5a5e-4abe-9ef5-2fbb89a4f582","Type":"ContainerStarted","Data":"4052cf005515a79863f374fe6ffc330783a00e287917f3d9d4495d44adb09911"}
Aug 05 17:21:16.327076 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:16.327036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:16.424219 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:16.424143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:16.438485 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:16.438451    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9" event={"ID":"84950fef-5a5e-4abe-9ef5-2fbb89a4f582","Type":"ContainerStarted","Data":"34c6f4b79a93b45647966fca5bd1383f0f1910b5b687e947106582a96366b62c"}
Aug 05 17:21:16.438798 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:16.438573    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:16.460202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:16.460165    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9" podStartSLOduration=1.108850007 podStartE2EDuration="2.46015177s" totalImagesPullingTime="1.351301763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:16.457045876 +0000 UTC m=+534.990230123" watchObservedRunningTime="2026-08-05 17:21:16.46015177 +0000 UTC m=+534.993336014"
Aug 05 17:21:20.419781 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:20.419740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:20.541448 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:20.541410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:24.091997 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:24.091917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:27.445342 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:27.445300    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-96fq9"
Aug 05 17:21:28.075421 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:28.075384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:28.157060 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:28.157008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:42.228905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:42.228875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:44.857236 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:44.857205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:45.195719 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:45.195689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:45.609675 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:45.609594    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:46.476380 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:46.476343    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:48.282799 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:48.282769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:48.389798 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:48.389765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:48.796688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:48.796652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:49.205541 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:49.205513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:49.910173 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:49.910144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:51.818271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:51.818233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:51.913585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:51.913534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:53.523273 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.523241    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"]
Aug 05 17:21:53.525609 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.525591    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:21:53.531524 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.531496    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8qdrk\""
Aug 05 17:21:53.532665 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.532646    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 05 17:21:53.548037 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.548018    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 05 17:21:53.557494 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.557471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"]
Aug 05 17:21:53.687422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.687390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbw2s\" (UniqueName: \"kubernetes.io/projected/6232136a-57c3-4491-bca8-80eab957578a-kube-api-access-cbw2s\") pod \"limitador-operator-controller-manager-774d94476f-gl8pz\" (UID: \"6232136a-57c3-4491-bca8-80eab957578a\") " pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:21:53.788716 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.788630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbw2s\" (UniqueName: \"kubernetes.io/projected/6232136a-57c3-4491-bca8-80eab957578a-kube-api-access-cbw2s\") pod \"limitador-operator-controller-manager-774d94476f-gl8pz\" (UID: \"6232136a-57c3-4491-bca8-80eab957578a\") " pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:21:53.807830 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.807800    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbw2s\" (UniqueName: \"kubernetes.io/projected/6232136a-57c3-4491-bca8-80eab957578a-kube-api-access-cbw2s\") pod \"limitador-operator-controller-manager-774d94476f-gl8pz\" (UID: \"6232136a-57c3-4491-bca8-80eab957578a\") " pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:21:53.835421 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.835393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:21:53.971183 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:53.971159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"]
Aug 05 17:21:53.973850 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:21:53.973819    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6232136a_57c3_4491_bca8_80eab957578a.slice/crio-826a079f882b8bba3a24e4907066a8c5baf63450474f305c154ab0995b965d97 WatchSource:0}: Error finding container 826a079f882b8bba3a24e4907066a8c5baf63450474f305c154ab0995b965d97: Status 404 returned error can't find the container with id 826a079f882b8bba3a24e4907066a8c5baf63450474f305c154ab0995b965d97
Aug 05 17:21:54.622623 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:54.622586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz" event={"ID":"6232136a-57c3-4491-bca8-80eab957578a","Type":"ContainerStarted","Data":"826a079f882b8bba3a24e4907066a8c5baf63450474f305c154ab0995b965d97"}
Aug 05 17:21:54.822527 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:54.822498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:54.926814 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:54.926726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:55.738538 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:55.738506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:55.829282 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:55.829240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:21:58.645289 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:58.645252    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz" event={"ID":"6232136a-57c3-4491-bca8-80eab957578a","Type":"ContainerStarted","Data":"cf6f0e6557ed07e2d7cf0a55d79795e90b4e7243cc4a59aa97564e9af407407e"}
Aug 05 17:21:58.645674 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:58.645389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:21:58.666659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:21:58.666609    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz" podStartSLOduration=1.748457895 podStartE2EDuration="5.666594272s" totalImagesPullingTime="3.918136377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:58.664064056 +0000 UTC m=+577.197248301" watchObservedRunningTime="2026-08-05 17:21:58.666594272 +0000 UTC m=+577.199778516"
Aug 05 17:22:00.005928 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:00.005893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:00.109468 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:00.109433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:00.213282 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:00.213208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:00.285322 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:00.285291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:00.396691 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:00.396653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:00.497389 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:00.497284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:08.289103 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:08.289069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:08.389177 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:08.389151    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:09.651880 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:09.651844    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-gl8pz"
Aug 05 17:22:11.312122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:11.312090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:11.409384 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:11.409349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:13.917075 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:13.917040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:14.039895 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:14.039864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:19.290004 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:19.289971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:19.390804 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:19.390766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:19.700191 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:19.700167    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:19.811950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:19.811914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:20.117010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:20.116929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:20.211425 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:20.211380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:21.869126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:21.869095    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:22:21.869551 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:21.869171    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:22:21.873912 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:21.873890    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:22:21.874031 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:21.873894    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:22:28.104157 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:28.104127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:28.202383 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:28.202353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:31.105080 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:31.105047    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:31.202177 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:31.202136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:31.733354 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:31.733301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:31.797759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:31.797733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:35.497714 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:35.497679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:35.612135 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:35.612106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:35.669236 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:35.669200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:36.356943 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.356908    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-bgkdc"]
Aug 05 17:22:36.360885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.360859    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:36.364373 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.364348    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h6p9s\""
Aug 05 17:22:36.365880 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.365853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bgkdc"]
Aug 05 17:22:36.383673 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.383631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:36.447207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.447178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5cpb\" (UniqueName: \"kubernetes.io/projected/784c8ee9-947e-45b3-b8c3-752e00243f10-kube-api-access-n5cpb\") pod \"authorino-b54df94b4-bgkdc\" (UID: \"784c8ee9-947e-45b3-b8c3-752e00243f10\") " pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:36.481159 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.481129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:36.548482 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.548450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cpb\" (UniqueName: \"kubernetes.io/projected/784c8ee9-947e-45b3-b8c3-752e00243f10-kube-api-access-n5cpb\") pod \"authorino-b54df94b4-bgkdc\" (UID: \"784c8ee9-947e-45b3-b8c3-752e00243f10\") " pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:36.556950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.556919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cpb\" (UniqueName: \"kubernetes.io/projected/784c8ee9-947e-45b3-b8c3-752e00243f10-kube-api-access-n5cpb\") pod \"authorino-b54df94b4-bgkdc\" (UID: \"784c8ee9-947e-45b3-b8c3-752e00243f10\") " pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:36.671689 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.671608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:36.800682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.800653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bgkdc"]
Aug 05 17:22:36.803629 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:22:36.803592    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod784c8ee9_947e_45b3_b8c3_752e00243f10.slice/crio-cc0a4f1bbf6ab7d119f62d7d9e42a6ad977a83640e0bb376851d28b2020d2e01 WatchSource:0}: Error finding container cc0a4f1bbf6ab7d119f62d7d9e42a6ad977a83640e0bb376851d28b2020d2e01: Status 404 returned error can't find the container with id cc0a4f1bbf6ab7d119f62d7d9e42a6ad977a83640e0bb376851d28b2020d2e01
Aug 05 17:22:36.821214 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:36.821184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bgkdc" event={"ID":"784c8ee9-947e-45b3-b8c3-752e00243f10","Type":"ContainerStarted","Data":"cc0a4f1bbf6ab7d119f62d7d9e42a6ad977a83640e0bb376851d28b2020d2e01"}
Aug 05 17:22:39.801957 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:39.801924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:40.075914 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:40.075839    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bgkdc"]
Aug 05 17:22:47.875721 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:47.875674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bgkdc" event={"ID":"784c8ee9-947e-45b3-b8c3-752e00243f10","Type":"ContainerStarted","Data":"6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99"}
Aug 05 17:22:47.876207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:47.875707    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-bgkdc" podUID="784c8ee9-947e-45b3-b8c3-752e00243f10" containerName="authorino" containerID="cri-o://6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99" gracePeriod=30
Aug 05 17:22:47.891212 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:47.891168    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-bgkdc" podStartSLOduration=0.935161226 podStartE2EDuration="11.891152246s" totalImagesPullingTime="10.95599102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:47.89099199 +0000 UTC m=+626.424176235" watchObservedRunningTime="2026-08-05 17:22:47.891152246 +0000 UTC m=+626.424336492"
Aug 05 17:22:48.128031 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.128006    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:48.148874 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.148839    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5cpb\" (UniqueName: \"kubernetes.io/projected/784c8ee9-947e-45b3-b8c3-752e00243f10-kube-api-access-n5cpb\") pod \"784c8ee9-947e-45b3-b8c3-752e00243f10\" (UID: \"784c8ee9-947e-45b3-b8c3-752e00243f10\") "
Aug 05 17:22:48.151539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.151506    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784c8ee9-947e-45b3-b8c3-752e00243f10-kube-api-access-n5cpb" (OuterVolumeSpecName: "kube-api-access-n5cpb") pod "784c8ee9-947e-45b3-b8c3-752e00243f10" (UID: "784c8ee9-947e-45b3-b8c3-752e00243f10"). InnerVolumeSpecName "kube-api-access-n5cpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:22:48.251084 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.251051    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5cpb\" (UniqueName: \"kubernetes.io/projected/784c8ee9-947e-45b3-b8c3-752e00243f10-kube-api-access-n5cpb\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:22:48.881305 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.881271    2562 generic.go:356] "Generic (PLEG): container finished" podID="784c8ee9-947e-45b3-b8c3-752e00243f10" containerID="6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99" exitCode=0
Aug 05 17:22:48.881779 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.881336    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bgkdc"
Aug 05 17:22:48.881779 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.881365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bgkdc" event={"ID":"784c8ee9-947e-45b3-b8c3-752e00243f10","Type":"ContainerDied","Data":"6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99"}
Aug 05 17:22:48.881779 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.881417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bgkdc" event={"ID":"784c8ee9-947e-45b3-b8c3-752e00243f10","Type":"ContainerDied","Data":"cc0a4f1bbf6ab7d119f62d7d9e42a6ad977a83640e0bb376851d28b2020d2e01"}
Aug 05 17:22:48.881779 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.881441    2562 scope.go:117] "RemoveContainer" containerID="6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99"
Aug 05 17:22:48.890051 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.890035    2562 scope.go:117] "RemoveContainer" containerID="6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99"
Aug 05 17:22:48.890290 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:22:48.890273    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99\": container with ID starting with 6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99 not found: ID does not exist" containerID="6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99"
Aug 05 17:22:48.890364 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.890297    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99"} err="failed to get container status \"6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99\": rpc error: code = NotFound desc = could not find container \"6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99\": container with ID starting with 6efff5064e33d353dbb13783746fd5cb197fa7b2d621f584bab0f4c8b7099d99 not found: ID does not exist"
Aug 05 17:22:48.908816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.908792    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bgkdc"]
Aug 05 17:22:48.910247 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:48.910228    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bgkdc"]
Aug 05 17:22:50.104507 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:50.104472    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784c8ee9-947e-45b3-b8c3-752e00243f10" path="/var/lib/kubelet/pods/784c8ee9-947e-45b3-b8c3-752e00243f10/volumes"
Aug 05 17:22:55.377423 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:55.377391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:56.113397 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:56.113365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:22:58.405263 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.405232    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7cf4765b85-j5frh"]
Aug 05 17:22:58.405660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.405622    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784c8ee9-947e-45b3-b8c3-752e00243f10" containerName="authorino"
Aug 05 17:22:58.405660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.405636    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="784c8ee9-947e-45b3-b8c3-752e00243f10" containerName="authorino"
Aug 05 17:22:58.405745 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.405711    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="784c8ee9-947e-45b3-b8c3-752e00243f10" containerName="authorino"
Aug 05 17:22:58.409919 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.409902    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.412887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.412863    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 05 17:22:58.413939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.413926    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h6p9s\""
Aug 05 17:22:58.418235 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.418211    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cf4765b85-j5frh"]
Aug 05 17:22:58.434261 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.434239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daf2446e-40a8-4461-949f-290b9d37c15e-tls-cert\") pod \"authorino-7cf4765b85-j5frh\" (UID: \"daf2446e-40a8-4461-949f-290b9d37c15e\") " pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.434351 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.434297    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvccm\" (UniqueName: \"kubernetes.io/projected/daf2446e-40a8-4461-949f-290b9d37c15e-kube-api-access-mvccm\") pod \"authorino-7cf4765b85-j5frh\" (UID: \"daf2446e-40a8-4461-949f-290b9d37c15e\") " pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.534666 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.534642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daf2446e-40a8-4461-949f-290b9d37c15e-tls-cert\") pod \"authorino-7cf4765b85-j5frh\" (UID: \"daf2446e-40a8-4461-949f-290b9d37c15e\") " pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.534760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.534695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvccm\" (UniqueName: \"kubernetes.io/projected/daf2446e-40a8-4461-949f-290b9d37c15e-kube-api-access-mvccm\") pod \"authorino-7cf4765b85-j5frh\" (UID: \"daf2446e-40a8-4461-949f-290b9d37c15e\") " pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.536950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.536931    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daf2446e-40a8-4461-949f-290b9d37c15e-tls-cert\") pod \"authorino-7cf4765b85-j5frh\" (UID: \"daf2446e-40a8-4461-949f-290b9d37c15e\") " pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.543385 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.543369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvccm\" (UniqueName: \"kubernetes.io/projected/daf2446e-40a8-4461-949f-290b9d37c15e-kube-api-access-mvccm\") pod \"authorino-7cf4765b85-j5frh\" (UID: \"daf2446e-40a8-4461-949f-290b9d37c15e\") " pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.719343 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.719298    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cf4765b85-j5frh"
Aug 05 17:22:58.847513 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.847265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cf4765b85-j5frh"]
Aug 05 17:22:58.850064 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:22:58.850042    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaf2446e_40a8_4461_949f_290b9d37c15e.slice/crio-9fe188ffb9fbf19bbd6f2b5da5e1cbae2ee08edf437fa1d65fe9813d6ba8b297 WatchSource:0}: Error finding container 9fe188ffb9fbf19bbd6f2b5da5e1cbae2ee08edf437fa1d65fe9813d6ba8b297: Status 404 returned error can't find the container with id 9fe188ffb9fbf19bbd6f2b5da5e1cbae2ee08edf437fa1d65fe9813d6ba8b297
Aug 05 17:22:58.928015 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:22:58.927988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cf4765b85-j5frh" event={"ID":"daf2446e-40a8-4461-949f-290b9d37c15e","Type":"ContainerStarted","Data":"9fe188ffb9fbf19bbd6f2b5da5e1cbae2ee08edf437fa1d65fe9813d6ba8b297"}
Aug 05 17:23:00.480356 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:00.480307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:00.538609 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:00.538575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:03.841591 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:03.841551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:03.942601 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:03.942572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:31.078535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:31.078499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cf4765b85-j5frh" event={"ID":"daf2446e-40a8-4461-949f-290b9d37c15e","Type":"ContainerStarted","Data":"eb82a7c362f79905b20f074e55e11fb1bf97b036096025ed326b078b78a93a61"}
Aug 05 17:23:31.095174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:31.095128    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7cf4765b85-j5frh" podStartSLOduration=1.4927372939999999 podStartE2EDuration="33.095114745s" totalImagesPullingTime="31.602377451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:23:31.094370604 +0000 UTC m=+669.627554851" watchObservedRunningTime="2026-08-05 17:23:31.095114745 +0000 UTC m=+669.628298991"
Aug 05 17:23:32.386976 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:32.386929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:32.483458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:32.483433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:32.550048 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:32.550022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:32.656395 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:32.656295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:32.917706 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:32.917631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:33.018921 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:33.018889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:33.086360 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:33.086334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:39.415529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:39.415491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:46.018849 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:46.018815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:23:46.119992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:23:46.119961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:01.751273 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:01.751236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:32.749980 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:32.749936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:32.828332 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:32.828280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:44.121155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:44.121126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:44.220194 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:44.220162    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:44.320636 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:44.320608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:52.940478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:52.940395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:53.028088 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:53.028051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:53.391230 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:53.391203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:53.455839 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:53.455810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:53.908142 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:53.908113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:24:53.972776 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:24:53.972753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:27.930516 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:27.930478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:29.324118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:29.324083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:39.966426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:39.966385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:40.067067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:40.067014    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:48.553211 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:48.553177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:48.614851 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:48.614818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:48.673542 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:48.673510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:49.048260 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.048224    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"]
Aug 05 17:26:49.051639 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.051619    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.054420 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.054396    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Aug 05 17:26:49.055788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.055770    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-gglx6\""
Aug 05 17:26:49.055902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.055828    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Aug 05 17:26:49.055902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.055858    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-v5w65\""
Aug 05 17:26:49.055902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.055862    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Aug 05 17:26:49.060651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.060631    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"]
Aug 05 17:26:49.072254 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.072234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:26:49.216074 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.216046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgxth\" (UniqueName: \"kubernetes.io/projected/40a64e47-d708-4b6f-9da0-5c8612533bff-kube-api-access-pgxth\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.216195 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.216078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a64e47-d708-4b6f-9da0-5c8612533bff-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.317122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.317053    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgxth\" (UniqueName: \"kubernetes.io/projected/40a64e47-d708-4b6f-9da0-5c8612533bff-kube-api-access-pgxth\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.317122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.317085    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a64e47-d708-4b6f-9da0-5c8612533bff-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.319278 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.319260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a64e47-d708-4b6f-9da0-5c8612533bff-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.325481 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.325460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgxth\" (UniqueName: \"kubernetes.io/projected/40a64e47-d708-4b6f-9da0-5c8612533bff-kube-api-access-pgxth\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.362927 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.362890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:49.508013 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.507985    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"]
Aug 05 17:26:49.509803 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:26:49.509775    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a64e47_d708_4b6f_9da0_5c8612533bff.slice/crio-cf2fc26ebd8077f47c7412d3f2cf68a0c3cf3301712860c38fef162b7a55a87b WatchSource:0}: Error finding container cf2fc26ebd8077f47c7412d3f2cf68a0c3cf3301712860c38fef162b7a55a87b: Status 404 returned error can't find the container with id cf2fc26ebd8077f47c7412d3f2cf68a0c3cf3301712860c38fef162b7a55a87b
Aug 05 17:26:49.511749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.511733    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:26:49.981066 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:49.981002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" event={"ID":"40a64e47-d708-4b6f-9da0-5c8612533bff","Type":"ContainerStarted","Data":"cf2fc26ebd8077f47c7412d3f2cf68a0c3cf3301712860c38fef162b7a55a87b"}
Aug 05 17:26:51.993430 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:51.993395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" event={"ID":"40a64e47-d708-4b6f-9da0-5c8612533bff","Type":"ContainerStarted","Data":"d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812"}
Aug 05 17:26:51.993849 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:51.993507    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:26:52.012032 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:26:52.011989    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" podStartSLOduration=0.620001883 podStartE2EDuration="3.011974442s" totalImagesPullingTime="2.391972559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:26:52.009969336 +0000 UTC m=+870.543153585" watchObservedRunningTime="2026-08-05 17:26:52.011974442 +0000 UTC m=+870.545158717"
Aug 05 17:27:00.434076 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.433999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:00.542782 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.542751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:00.678823 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.678793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:00.717301 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.717260    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"]
Aug 05 17:27:00.721758 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.721738    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:00.725874 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.725849    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"openshift-service-ca.crt\""
Aug 05 17:27:00.726415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.726303    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa-dockercfg-dddzl\""
Aug 05 17:27:00.726805 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.726788    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Aug 05 17:27:00.727150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.727133    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"default-dockercfg-q5sfd\""
Aug 05 17:27:00.727485 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.727467    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"kube-root-ca.crt\""
Aug 05 17:27:00.736418 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.736396    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"]
Aug 05 17:27:00.756579 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.756550    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:00.812943 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.812912    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71c935a-5e73-44da-b36e-2464100e34b5-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:00.813079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.813057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6klf\" (UniqueName: \"kubernetes.io/projected/f71c935a-5e73-44da-b36e-2464100e34b5-kube-api-access-v6klf\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:00.914288 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.914257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6klf\" (UniqueName: \"kubernetes.io/projected/f71c935a-5e73-44da-b36e-2464100e34b5-kube-api-access-v6klf\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:00.914433 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.914293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71c935a-5e73-44da-b36e-2464100e34b5-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:00.916827 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.916806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71c935a-5e73-44da-b36e-2464100e34b5-tls-certs\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:00.922847 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:00.922822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6klf\" (UniqueName: \"kubernetes.io/projected/f71c935a-5e73-44da-b36e-2464100e34b5-kube-api-access-v6klf\") pod \"llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:01.037132 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:01.037069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:01.166454 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:01.166432    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"]
Aug 05 17:27:01.168788 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:27:01.168758    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71c935a_5e73_44da_b36e_2464100e34b5.slice/crio-368b54bb1dede61e54df6e13dc7d93d62da934738800e906cfa8d864d1de6dfa WatchSource:0}: Error finding container 368b54bb1dede61e54df6e13dc7d93d62da934738800e906cfa8d864d1de6dfa: Status 404 returned error can't find the container with id 368b54bb1dede61e54df6e13dc7d93d62da934738800e906cfa8d864d1de6dfa
Aug 05 17:27:02.050006 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:02.049972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" event={"ID":"f71c935a-5e73-44da-b36e-2464100e34b5","Type":"ContainerStarted","Data":"1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9"}
Aug 05 17:27:02.050503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:02.050028    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" event={"ID":"f71c935a-5e73-44da-b36e-2464100e34b5","Type":"ContainerStarted","Data":"368b54bb1dede61e54df6e13dc7d93d62da934738800e906cfa8d864d1de6dfa"}
Aug 05 17:27:02.050503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:02.050120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:02.068924 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:02.068880    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" podStartSLOduration=2.068867313 podStartE2EDuration="2.068867313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:02.067027887 +0000 UTC m=+880.600212133" watchObservedRunningTime="2026-08-05 17:27:02.068867313 +0000 UTC m=+880.602051557"
Aug 05 17:27:07.733709 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:07.733677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:20.099509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:20.099478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:21.898472 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:21.898441    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:27:21.900696 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:21.900677    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:27:21.903354 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:21.903337    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:27:21.905529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:21.905515    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:27:23.005541 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:23.005512    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:27:30.149437 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:30.149403    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:30.150791 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:30.150766    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"]
Aug 05 17:27:30.245400 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:30.245373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:33.005902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:33.005863    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:27:33.056290 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:33.056262    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:27:41.992677 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:41.992646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:43.006336 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:43.006277    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:27:45.170780 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.170743    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerName="main" containerID="cri-o://d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812" gracePeriod=45
Aug 05 17:27:45.417180 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.417154    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:27:45.581586 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.581561    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgxth\" (UniqueName: \"kubernetes.io/projected/40a64e47-d708-4b6f-9da0-5c8612533bff-kube-api-access-pgxth\") pod \"40a64e47-d708-4b6f-9da0-5c8612533bff\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") "
Aug 05 17:27:45.581748 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.581599    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a64e47-d708-4b6f-9da0-5c8612533bff-tls-certs\") pod \"40a64e47-d708-4b6f-9da0-5c8612533bff\" (UID: \"40a64e47-d708-4b6f-9da0-5c8612533bff\") "
Aug 05 17:27:45.583735 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.583696    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a64e47-d708-4b6f-9da0-5c8612533bff-kube-api-access-pgxth" (OuterVolumeSpecName: "kube-api-access-pgxth") pod "40a64e47-d708-4b6f-9da0-5c8612533bff" (UID: "40a64e47-d708-4b6f-9da0-5c8612533bff"). InnerVolumeSpecName "kube-api-access-pgxth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:27:45.583838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.583782    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40a64e47-d708-4b6f-9da0-5c8612533bff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40a64e47-d708-4b6f-9da0-5c8612533bff" (UID: "40a64e47-d708-4b6f-9da0-5c8612533bff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:27:45.682726 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.682698    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgxth\" (UniqueName: \"kubernetes.io/projected/40a64e47-d708-4b6f-9da0-5c8612533bff-kube-api-access-pgxth\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:27:45.682726 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:45.682720    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a64e47-d708-4b6f-9da0-5c8612533bff-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:27:46.258595 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.258560    2562 generic.go:356] "Generic (PLEG): container finished" podID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerID="d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812" exitCode=0
Aug 05 17:27:46.258953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.258615    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"
Aug 05 17:27:46.258953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.258643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" event={"ID":"40a64e47-d708-4b6f-9da0-5c8612533bff","Type":"ContainerDied","Data":"d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812"}
Aug 05 17:27:46.258953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.258697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp" event={"ID":"40a64e47-d708-4b6f-9da0-5c8612533bff","Type":"ContainerDied","Data":"cf2fc26ebd8077f47c7412d3f2cf68a0c3cf3301712860c38fef162b7a55a87b"}
Aug 05 17:27:46.258953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.258719    2562 scope.go:117] "RemoveContainer" containerID="d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812"
Aug 05 17:27:46.267150 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.267119    2562 scope.go:117] "RemoveContainer" containerID="d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812"
Aug 05 17:27:46.267403 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:27:46.267385    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812\": container with ID starting with d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812 not found: ID does not exist" containerID="d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812"
Aug 05 17:27:46.267465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.267410    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812"} err="failed to get container status \"d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812\": rpc error: code = NotFound desc = could not find container \"d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812\": container with ID starting with d580ead1e912336cf561a5b4afe26cf4bfb5eecb83909b9dc414e1ddf74f3812 not found: ID does not exist"
Aug 05 17:27:46.276149 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.276097    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"]
Aug 05 17:27:46.277807 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:46.277789    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-tgskp"]
Aug 05 17:27:48.102045 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:48.102012    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" path="/var/lib/kubelet/pods/40a64e47-d708-4b6f-9da0-5c8612533bff/volumes"
Aug 05 17:27:49.860568 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:49.860523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:49.959477 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:49.959447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:55.951499 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:55.951451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:57.720400 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:57.720359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:27:57.822304 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:27:57.822267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:00.403558 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.403464    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:00.646011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.645982    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"]
Aug 05 17:28:00.646372 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.646357    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerName="main"
Aug 05 17:28:00.646450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.646374    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerName="main"
Aug 05 17:28:00.646509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.646458    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a64e47-d708-4b6f-9da0-5c8612533bff" containerName="main"
Aug 05 17:28:00.649614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.649593    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.653631 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.653506    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-z4rm5\""
Aug 05 17:28:00.653631 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.653524    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Aug 05 17:28:00.653631 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.653505    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Aug 05 17:28:00.653631 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.653561    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-epp-sa-dockercfg-ps29t\""
Aug 05 17:28:00.654075 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.654056    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Aug 05 17:28:00.662193 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.662171    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"]
Aug 05 17:28:00.671722 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.671700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:00.694038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.694014    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqwzl\" (UniqueName: \"kubernetes.io/projected/5756e1a8-4b57-4df6-863c-fc7ed16038c1-kube-api-access-zqwzl\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.694144 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.694055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5756e1a8-4b57-4df6-863c-fc7ed16038c1-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.794767 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.794740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqwzl\" (UniqueName: \"kubernetes.io/projected/5756e1a8-4b57-4df6-863c-fc7ed16038c1-kube-api-access-zqwzl\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.794890 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.794784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5756e1a8-4b57-4df6-863c-fc7ed16038c1-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.797181 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.797156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5756e1a8-4b57-4df6-863c-fc7ed16038c1-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.802961 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.802940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqwzl\" (UniqueName: \"kubernetes.io/projected/5756e1a8-4b57-4df6-863c-fc7ed16038c1-kube-api-access-zqwzl\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:00.964525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:00.964498    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:01.100205 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:01.100173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"]
Aug 05 17:28:01.103660 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:28:01.103626    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5756e1a8_4b57_4df6_863c_fc7ed16038c1.slice/crio-8c969e7ab293f76e000bb45a2c8be590b33414041d25e6b4c0b992e3d27b528d WatchSource:0}: Error finding container 8c969e7ab293f76e000bb45a2c8be590b33414041d25e6b4c0b992e3d27b528d: Status 404 returned error can't find the container with id 8c969e7ab293f76e000bb45a2c8be590b33414041d25e6b4c0b992e3d27b528d
Aug 05 17:28:01.328958 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:01.328866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" event={"ID":"5756e1a8-4b57-4df6-863c-fc7ed16038c1","Type":"ContainerStarted","Data":"21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880"}
Aug 05 17:28:01.328958 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:01.328928    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" event={"ID":"5756e1a8-4b57-4df6-863c-fc7ed16038c1","Type":"ContainerStarted","Data":"8c969e7ab293f76e000bb45a2c8be590b33414041d25e6b4c0b992e3d27b528d"}
Aug 05 17:28:01.329136 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:01.329040    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:01.351656 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:01.351603    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" podStartSLOduration=1.351589616 podStartE2EDuration="1.351589616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:01.350653712 +0000 UTC m=+939.883837960" watchObservedRunningTime="2026-08-05 17:28:01.351589616 +0000 UTC m=+939.884773861"
Aug 05 17:28:02.457036 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:02.457004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:02.515566 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:02.515534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:06.503908 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:06.503872    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:32.335140 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:32.335113    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:39.164371 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:39.164334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:39.173299 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:39.173259    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"]
Aug 05 17:28:39.261259 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:39.261232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:42.335122 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:42.335083    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:28:48.328942 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:48.328863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:52.334768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:52.334724    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:28:54.194290 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.194244    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerName="main" containerID="cri-o://21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880" gracePeriod=45
Aug 05 17:28:54.444820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.444764    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:54.588640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.588606    2562 generic.go:356] "Generic (PLEG): container finished" podID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerID="21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880" exitCode=0
Aug 05 17:28:54.588803 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.588655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"
Aug 05 17:28:54.588803 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.588696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" event={"ID":"5756e1a8-4b57-4df6-863c-fc7ed16038c1","Type":"ContainerDied","Data":"21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880"}
Aug 05 17:28:54.588803 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.588751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48" event={"ID":"5756e1a8-4b57-4df6-863c-fc7ed16038c1","Type":"ContainerDied","Data":"8c969e7ab293f76e000bb45a2c8be590b33414041d25e6b4c0b992e3d27b528d"}
Aug 05 17:28:54.588803 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.588775    2562 scope.go:117] "RemoveContainer" containerID="21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880"
Aug 05 17:28:54.596923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.596905    2562 scope.go:117] "RemoveContainer" containerID="21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880"
Aug 05 17:28:54.597186 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:28:54.597162    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880\": container with ID starting with 21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880 not found: ID does not exist" containerID="21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880"
Aug 05 17:28:54.597253 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.597194    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880"} err="failed to get container status \"21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880\": rpc error: code = NotFound desc = could not find container \"21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880\": container with ID starting with 21a5f2cecfc19cbf168fb3d728a312d8e4ea6fd1fe3970ec937a23518de49880 not found: ID does not exist"
Aug 05 17:28:54.626570 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.626553    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqwzl\" (UniqueName: \"kubernetes.io/projected/5756e1a8-4b57-4df6-863c-fc7ed16038c1-kube-api-access-zqwzl\") pod \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") "
Aug 05 17:28:54.626703 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.626691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5756e1a8-4b57-4df6-863c-fc7ed16038c1-tls-certs\") pod \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\" (UID: \"5756e1a8-4b57-4df6-863c-fc7ed16038c1\") "
Aug 05 17:28:54.628634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.628608    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5756e1a8-4b57-4df6-863c-fc7ed16038c1-kube-api-access-zqwzl" (OuterVolumeSpecName: "kube-api-access-zqwzl") pod "5756e1a8-4b57-4df6-863c-fc7ed16038c1" (UID: "5756e1a8-4b57-4df6-863c-fc7ed16038c1"). InnerVolumeSpecName "kube-api-access-zqwzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:28:54.628728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.628611    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5756e1a8-4b57-4df6-863c-fc7ed16038c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5756e1a8-4b57-4df6-863c-fc7ed16038c1" (UID: "5756e1a8-4b57-4df6-863c-fc7ed16038c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:28:54.728220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.728158    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5756e1a8-4b57-4df6-863c-fc7ed16038c1-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:28:54.728220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.728184    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqwzl\" (UniqueName: \"kubernetes.io/projected/5756e1a8-4b57-4df6-863c-fc7ed16038c1-kube-api-access-zqwzl\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:28:54.911205 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.911178    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"]
Aug 05 17:28:54.916712 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:54.916687    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-8lb48"]
Aug 05 17:28:56.109391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:56.109355    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" path="/var/lib/kubelet/pods/5756e1a8-4b57-4df6-863c-fc7ed16038c1/volumes"
Aug 05 17:28:59.001979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:59.001944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:59.104874 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:59.104843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:59.209754 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:59.209719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:28:59.301029 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:28:59.300953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:03.417135 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:03.417103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:05.429473 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:05.429440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:11.542471 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:11.542422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:11.601821 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:11.601788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:26.653128 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:26.653093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:26.753720 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:26.753691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:40.113333 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.113289    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"]
Aug 05 17:29:40.113764 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.113710    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerName="main"
Aug 05 17:29:40.113764 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.113725    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerName="main"
Aug 05 17:29:40.113852 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.113777    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5756e1a8-4b57-4df6-863c-fc7ed16038c1" containerName="main"
Aug 05 17:29:40.117485 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.117468    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.120470 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.120449    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\""
Aug 05 17:29:40.121687 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.121667    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"default-dockercfg-6v2tq\""
Aug 05 17:29:40.121799 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.121709    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\""
Aug 05 17:29:40.121799 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.121709    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs\""
Aug 05 17:29:40.127174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.127153    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"]
Aug 05 17:29:40.156842 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.156818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:40.179957 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.179933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.180082 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.179964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.180082 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.179982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/327d56b3-492d-4c6f-85fe-ce5f70d69275-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.180184 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.180165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.180257 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.180239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.180387 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.180368    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j9fd\" (UniqueName: \"kubernetes.io/projected/327d56b3-492d-4c6f-85fe-ce5f70d69275-kube-api-access-9j9fd\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.280766 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.280739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j9fd\" (UniqueName: \"kubernetes.io/projected/327d56b3-492d-4c6f-85fe-ce5f70d69275-kube-api-access-9j9fd\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.280947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.280791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.280947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.280811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.280947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.280922    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/327d56b3-492d-4c6f-85fe-ce5f70d69275-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.281151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.280976    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.281151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.281027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.281151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.281144    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.281370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.281341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.281502 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.281418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.282925 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.282900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.283229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.283214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/327d56b3-492d-4c6f-85fe-ce5f70d69275-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.288627 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.288604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j9fd\" (UniqueName: \"kubernetes.io/projected/327d56b3-492d-4c6f-85fe-ce5f70d69275-kube-api-access-9j9fd\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.428574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.428508    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:40.565343 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.565300    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"]
Aug 05 17:29:40.567803 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:29:40.567775    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327d56b3_492d_4c6f_85fe_ce5f70d69275.slice/crio-23616a1e4a4406b63e3c975d0d19deb9d9eaa14d15ec202915e33f83f013c609 WatchSource:0}: Error finding container 23616a1e4a4406b63e3c975d0d19deb9d9eaa14d15ec202915e33f83f013c609: Status 404 returned error can't find the container with id 23616a1e4a4406b63e3c975d0d19deb9d9eaa14d15ec202915e33f83f013c609
Aug 05 17:29:40.803296 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:40.803264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" event={"ID":"327d56b3-492d-4c6f-85fe-ce5f70d69275","Type":"ContainerStarted","Data":"23616a1e4a4406b63e3c975d0d19deb9d9eaa14d15ec202915e33f83f013c609"}
Aug 05 17:29:42.341791 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.341756    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:42.442051 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.441969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:42.542202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.542168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:42.615822 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.615786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:42.681971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.681940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:42.814723 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.814688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" event={"ID":"327d56b3-492d-4c6f-85fe-ce5f70d69275","Type":"ContainerStarted","Data":"2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"}
Aug 05 17:29:42.832752 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:42.832710    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" podStartSLOduration=1.240743013 podStartE2EDuration="2.832697539s" totalImagesPullingTime="1.591954526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:42.831706549 +0000 UTC m=+1041.364890795" watchObservedRunningTime="2026-08-05 17:29:42.832697539 +0000 UTC m=+1041.365881784"
Aug 05 17:29:43.428655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:43.428622    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:46.191649 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:46.191622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:46.788663 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:46.788632    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"]
Aug 05 17:29:46.797676 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:29:46.797641    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:29:46.797676 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:29:46.797676 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:29:46.797676 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956" cmd=["/bin/sleep","15"]
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:46.797690    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" message="" err=<
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:29:46.797705    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" podUID="327d56b3-492d-4c6f-85fe-ce5f70d69275" containerName="main" containerID="cri-o://2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"
Aug 05 17:29:46.797904 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:46.797743    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" podUID="327d56b3-492d-4c6f-85fe-ce5f70d69275" containerName="main" containerID="cri-o://2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956" gracePeriod=60
Aug 05 17:29:46.824426 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:46.824383    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:47.048052 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.047999    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:47.142151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142125    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j9fd\" (UniqueName: \"kubernetes.io/projected/327d56b3-492d-4c6f-85fe-ce5f70d69275-kube-api-access-9j9fd\") pod \"327d56b3-492d-4c6f-85fe-ce5f70d69275\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") "
Aug 05 17:29:47.142346 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142167    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-model-cache\") pod \"327d56b3-492d-4c6f-85fe-ce5f70d69275\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") "
Aug 05 17:29:47.142422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142363    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-dshm\") pod \"327d56b3-492d-4c6f-85fe-ce5f70d69275\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") "
Aug 05 17:29:47.142422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142393    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-model-cache" (OuterVolumeSpecName: "model-cache") pod "327d56b3-492d-4c6f-85fe-ce5f70d69275" (UID: "327d56b3-492d-4c6f-85fe-ce5f70d69275"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:29:47.142536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142405    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-home\") pod \"327d56b3-492d-4c6f-85fe-ce5f70d69275\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") "
Aug 05 17:29:47.142536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142464    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/327d56b3-492d-4c6f-85fe-ce5f70d69275-tls-certs\") pod \"327d56b3-492d-4c6f-85fe-ce5f70d69275\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") "
Aug 05 17:29:47.142536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142503    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-tmp-dir\") pod \"327d56b3-492d-4c6f-85fe-ce5f70d69275\" (UID: \"327d56b3-492d-4c6f-85fe-ce5f70d69275\") "
Aug 05 17:29:47.142690 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142625    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-home" (OuterVolumeSpecName: "home") pod "327d56b3-492d-4c6f-85fe-ce5f70d69275" (UID: "327d56b3-492d-4c6f-85fe-ce5f70d69275"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:29:47.142841 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "327d56b3-492d-4c6f-85fe-ce5f70d69275" (UID: "327d56b3-492d-4c6f-85fe-ce5f70d69275"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:29:47.142841 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142832    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:29:47.143193 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.142850    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:29:47.144410 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.144387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-dshm" (OuterVolumeSpecName: "dshm") pod "327d56b3-492d-4c6f-85fe-ce5f70d69275" (UID: "327d56b3-492d-4c6f-85fe-ce5f70d69275"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:29:47.144508 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.144384    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/327d56b3-492d-4c6f-85fe-ce5f70d69275-kube-api-access-9j9fd" (OuterVolumeSpecName: "kube-api-access-9j9fd") pod "327d56b3-492d-4c6f-85fe-ce5f70d69275" (UID: "327d56b3-492d-4c6f-85fe-ce5f70d69275"). InnerVolumeSpecName "kube-api-access-9j9fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:29:47.144508 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.144448    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/327d56b3-492d-4c6f-85fe-ce5f70d69275-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "327d56b3-492d-4c6f-85fe-ce5f70d69275" (UID: "327d56b3-492d-4c6f-85fe-ce5f70d69275"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:29:47.243565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.243536    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j9fd\" (UniqueName: \"kubernetes.io/projected/327d56b3-492d-4c6f-85fe-ce5f70d69275-kube-api-access-9j9fd\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:29:47.243565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.243563    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:29:47.243988 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.243579    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/327d56b3-492d-4c6f-85fe-ce5f70d69275-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:29:47.243988 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.243603    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/327d56b3-492d-4c6f-85fe-ce5f70d69275-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:29:47.366130 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.366029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:47.842115 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.842083    2562 generic.go:356] "Generic (PLEG): container finished" podID="327d56b3-492d-4c6f-85fe-ce5f70d69275" containerID="2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956" exitCode=0
Aug 05 17:29:47.842277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.842128    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"
Aug 05 17:29:47.842277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.842166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" event={"ID":"327d56b3-492d-4c6f-85fe-ce5f70d69275","Type":"ContainerDied","Data":"2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"}
Aug 05 17:29:47.842277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.842219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c" event={"ID":"327d56b3-492d-4c6f-85fe-ce5f70d69275","Type":"ContainerDied","Data":"23616a1e4a4406b63e3c975d0d19deb9d9eaa14d15ec202915e33f83f013c609"}
Aug 05 17:29:47.842277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.842251    2562 scope.go:117] "RemoveContainer" containerID="2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"
Aug 05 17:29:47.851024 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.851009    2562 scope.go:117] "RemoveContainer" containerID="2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"
Aug 05 17:29:47.851269 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:29:47.851248    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956\": container with ID starting with 2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956 not found: ID does not exist" containerID="2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"
Aug 05 17:29:47.851340 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.851278    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956"} err="failed to get container status \"2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956\": rpc error: code = NotFound desc = could not find container \"2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956\": container with ID starting with 2d33ccf6394d33b9a7163c48b15b5242d804f2862ca9081365ba9f662946e956 not found: ID does not exist"
Aug 05 17:29:47.862927 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.862898    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"]
Aug 05 17:29:47.865018 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:47.864993    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-49t5c"]
Aug 05 17:29:48.102264 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:48.102187    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="327d56b3-492d-4c6f-85fe-ce5f70d69275" path="/var/lib/kubelet/pods/327d56b3-492d-4c6f-85fe-ce5f70d69275/volumes"
Aug 05 17:29:52.448526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:52.448487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:53.568123 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:53.568095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:58.673226 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:58.673194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:59.303270 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:59.303241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:29:59.404960 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:29:59.404930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:05.007852 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.007817    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"]
Aug 05 17:30:05.008228 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.008209    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="327d56b3-492d-4c6f-85fe-ce5f70d69275" containerName="main"
Aug 05 17:30:05.008228 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.008221    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="327d56b3-492d-4c6f-85fe-ce5f70d69275" containerName="main"
Aug 05 17:30:05.008302 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.008289    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="327d56b3-492d-4c6f-85fe-ce5f70d69275" containerName="main"
Aug 05 17:30:05.011401 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.011384    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.014127 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.014107    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Aug 05 17:30:05.015275 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.015253    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Aug 05 17:30:05.015407 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.015266    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Aug 05 17:30:05.015407 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.015269    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"default-dockercfg-vltn7\""
Aug 05 17:30:05.021114 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.021090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"]
Aug 05 17:30:05.093739 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.093708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.093882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.093746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.093882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.093774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.093882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.093844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7xl5\" (UniqueName: \"kubernetes.io/projected/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-kube-api-access-f7xl5\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.094027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.093911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.094027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.093936    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.098513 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.098488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:05.194808 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.194775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.194981 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.194817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.194981 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.194865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7xl5\" (UniqueName: \"kubernetes.io/projected/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-kube-api-access-f7xl5\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.194981 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.194935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.194981 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.194967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.195199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.195086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.195445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.195418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.195537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.195490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.195537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.195522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.197146 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.197129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.197284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.197266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.203113 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.203090    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7xl5\" (UniqueName: \"kubernetes.io/projected/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-kube-api-access-f7xl5\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.322689 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.322587    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:05.453682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.453654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"]
Aug 05 17:30:05.456372 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:30:05.456348    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8bfcaf_1779_4e2f_b98e_51fde04ed0a2.slice/crio-b25304199b331c209ba6082fe49c7df097da54032386ac8949e9628dae70645e WatchSource:0}: Error finding container b25304199b331c209ba6082fe49c7df097da54032386ac8949e9628dae70645e: Status 404 returned error can't find the container with id b25304199b331c209ba6082fe49c7df097da54032386ac8949e9628dae70645e
Aug 05 17:30:05.927602 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.927563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" event={"ID":"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2","Type":"ContainerStarted","Data":"e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"}
Aug 05 17:30:05.927793 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.927639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" event={"ID":"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2","Type":"ContainerStarted","Data":"b25304199b331c209ba6082fe49c7df097da54032386ac8949e9628dae70645e"}
Aug 05 17:30:05.947996 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:05.947947    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" podStartSLOduration=1.947931774 podStartE2EDuration="1.947931774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:05.946295193 +0000 UTC m=+1064.479479438" watchObservedRunningTime="2026-08-05 17:30:05.947931774 +0000 UTC m=+1064.481116019"
Aug 05 17:30:06.323591 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:06.323557    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:07.453493 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:07.453451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:07.571506 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:07.571467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:07.606677 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:07.606641    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"]
Aug 05 17:30:07.650367 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:07.650340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:08.244088 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:08.244055    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"]
Aug 05 17:30:08.254396 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:08.254372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:08.950007 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:30:08.949969    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:30:08.950007 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:30:08.950007 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:30:08.950007 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a" cmd=["/bin/sleep","15"]
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:08.950017    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" message="" err=<
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:30:08.950034    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" podUID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" containerName="main" containerID="cri-o://e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"
Aug 05 17:30:08.950520 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:08.950086    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" podUID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" containerName="main" containerID="cri-o://e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a" gracePeriod=60
Aug 05 17:30:09.189634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.189604    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:09.231634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231549    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7xl5\" (UniqueName: \"kubernetes.io/projected/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-kube-api-access-f7xl5\") pod \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") "
Aug 05 17:30:09.231634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tls-certs\") pod \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") "
Aug 05 17:30:09.231832 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231668    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-model-cache\") pod \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") "
Aug 05 17:30:09.231832 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231692    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tmp-dir\") pod \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") "
Aug 05 17:30:09.231832 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231772    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-home\") pod \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") "
Aug 05 17:30:09.231832 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231811    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-dshm\") pod \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\" (UID: \"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2\") "
Aug 05 17:30:09.232044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231918    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" (UID: "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:09.232044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.231979    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" (UID: "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:09.232141 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.232126    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:09.232194 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.232143    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:09.232194 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.232148    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-home" (OuterVolumeSpecName: "home") pod "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" (UID: "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:09.234062 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.234037    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-dshm" (OuterVolumeSpecName: "dshm") pod "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" (UID: "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:09.234202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.234117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-kube-api-access-f7xl5" (OuterVolumeSpecName: "kube-api-access-f7xl5") pod "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" (UID: "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2"). InnerVolumeSpecName "kube-api-access-f7xl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:30:09.234202 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.234123    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" (UID: "af8bfcaf-1779-4e2f-b98e-51fde04ed0a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:30:09.332564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.332535    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:09.332564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.332559    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:09.332564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.332569    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7xl5\" (UniqueName: \"kubernetes.io/projected/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-kube-api-access-f7xl5\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:09.332779 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.332577    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:09.946547 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.946513    2562 generic.go:356] "Generic (PLEG): container finished" podID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" containerID="e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a" exitCode=0
Aug 05 17:30:09.946715 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.946564    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"
Aug 05 17:30:09.946715 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.946598    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" event={"ID":"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2","Type":"ContainerDied","Data":"e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"}
Aug 05 17:30:09.946715 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.946650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x" event={"ID":"af8bfcaf-1779-4e2f-b98e-51fde04ed0a2","Type":"ContainerDied","Data":"b25304199b331c209ba6082fe49c7df097da54032386ac8949e9628dae70645e"}
Aug 05 17:30:09.946715 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.946672    2562 scope.go:117] "RemoveContainer" containerID="e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"
Aug 05 17:30:09.956216 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.956031    2562 scope.go:117] "RemoveContainer" containerID="e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"
Aug 05 17:30:09.956478 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:30:09.956288    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a\": container with ID starting with e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a not found: ID does not exist" containerID="e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"
Aug 05 17:30:09.956478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.956325    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a"} err="failed to get container status \"e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a\": rpc error: code = NotFound desc = could not find container \"e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a\": container with ID starting with e3a6256266cc49567e90f666d812dc79e65753a004524db7ee63d8ed5ad1065a not found: ID does not exist"
Aug 05 17:30:09.972617 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.972595    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"]
Aug 05 17:30:09.976652 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:09.976634    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfrtx8x"]
Aug 05 17:30:10.102918 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:10.102887    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" path="/var/lib/kubelet/pods/af8bfcaf-1779-4e2f-b98e-51fde04ed0a2/volumes"
Aug 05 17:30:12.541346 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:12.541297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:13.013324 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.013282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:13.046979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.046950    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"]
Aug 05 17:30:13.047305 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.047293    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" containerName="main"
Aug 05 17:30:13.047384 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.047307    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" containerName="main"
Aug 05 17:30:13.047421 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.047403    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="af8bfcaf-1779-4e2f-b98e-51fde04ed0a2" containerName="main"
Aug 05 17:30:13.053734 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.053714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.056100 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.056057    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" podUID="f71c935a-5e73-44da-b36e-2464100e34b5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:30:13.059425 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.059403    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-lm2kn\""
Aug 05 17:30:13.059539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.059480    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Aug 05 17:30:13.060515 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.060496    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Aug 05 17:30:13.061004 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.060679    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Aug 05 17:30:13.061079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.061026    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-58dwq\""
Aug 05 17:30:13.068415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.068393    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"]
Aug 05 17:30:13.089046 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.089008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:13.165222 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.165198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017b6c68-1344-4d9c-891a-99daeebc10d8-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.165348 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.165241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6c9x\" (UniqueName: \"kubernetes.io/projected/017b6c68-1344-4d9c-891a-99daeebc10d8-kube-api-access-k6c9x\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.266537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.266470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017b6c68-1344-4d9c-891a-99daeebc10d8-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.266537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.266506    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6c9x\" (UniqueName: \"kubernetes.io/projected/017b6c68-1344-4d9c-891a-99daeebc10d8-kube-api-access-k6c9x\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.268806 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.268785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017b6c68-1344-4d9c-891a-99daeebc10d8-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.274614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.274590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6c9x\" (UniqueName: \"kubernetes.io/projected/017b6c68-1344-4d9c-891a-99daeebc10d8-kube-api-access-k6c9x\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.364677 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.364646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.507640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.507611    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"]
Aug 05 17:30:13.509110 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:30:13.509084    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017b6c68_1344_4d9c_891a_99daeebc10d8.slice/crio-331cea3eb392218e6c323ff4e1b95281128dafec77aff14574084d5fb7b94fc7 WatchSource:0}: Error finding container 331cea3eb392218e6c323ff4e1b95281128dafec77aff14574084d5fb7b94fc7: Status 404 returned error can't find the container with id 331cea3eb392218e6c323ff4e1b95281128dafec77aff14574084d5fb7b94fc7
Aug 05 17:30:13.969123 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.969081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" event={"ID":"017b6c68-1344-4d9c-891a-99daeebc10d8","Type":"ContainerStarted","Data":"fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f"}
Aug 05 17:30:13.969624 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.969301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:13.969624 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.969368    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" event={"ID":"017b6c68-1344-4d9c-891a-99daeebc10d8","Type":"ContainerStarted","Data":"331cea3eb392218e6c323ff4e1b95281128dafec77aff14574084d5fb7b94fc7"}
Aug 05 17:30:13.987374 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:13.987303    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" podStartSLOduration=0.98728677 podStartE2EDuration="987.28677ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:13.986463433 +0000 UTC m=+1072.519647711" watchObservedRunningTime="2026-08-05 17:30:13.98728677 +0000 UTC m=+1072.520471015"
Aug 05 17:30:18.699066 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:18.699033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:19.132535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:19.132460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:22.625851 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:22.625817    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" podUID="f71c935a-5e73-44da-b36e-2464100e34b5" containerName="main" containerID="cri-o://1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9" gracePeriod=45
Aug 05 17:30:22.865686 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:22.865661    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:30:22.948303 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:22.948278    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6klf\" (UniqueName: \"kubernetes.io/projected/f71c935a-5e73-44da-b36e-2464100e34b5-kube-api-access-v6klf\") pod \"f71c935a-5e73-44da-b36e-2464100e34b5\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") "
Aug 05 17:30:22.948450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:22.948435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71c935a-5e73-44da-b36e-2464100e34b5-tls-certs\") pod \"f71c935a-5e73-44da-b36e-2464100e34b5\" (UID: \"f71c935a-5e73-44da-b36e-2464100e34b5\") "
Aug 05 17:30:22.950417 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:22.950392    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f71c935a-5e73-44da-b36e-2464100e34b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f71c935a-5e73-44da-b36e-2464100e34b5" (UID: "f71c935a-5e73-44da-b36e-2464100e34b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:30:22.950517 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:22.950496    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f71c935a-5e73-44da-b36e-2464100e34b5-kube-api-access-v6klf" (OuterVolumeSpecName: "kube-api-access-v6klf") pod "f71c935a-5e73-44da-b36e-2464100e34b5" (UID: "f71c935a-5e73-44da-b36e-2464100e34b5"). InnerVolumeSpecName "kube-api-access-v6klf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:30:23.011987 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.011959    2562 generic.go:356] "Generic (PLEG): container finished" podID="f71c935a-5e73-44da-b36e-2464100e34b5" containerID="1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9" exitCode=0
Aug 05 17:30:23.012117 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.012008    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"
Aug 05 17:30:23.012117 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.012040    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" event={"ID":"f71c935a-5e73-44da-b36e-2464100e34b5","Type":"ContainerDied","Data":"1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9"}
Aug 05 17:30:23.012238 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.012123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn" event={"ID":"f71c935a-5e73-44da-b36e-2464100e34b5","Type":"ContainerDied","Data":"368b54bb1dede61e54df6e13dc7d93d62da934738800e906cfa8d864d1de6dfa"}
Aug 05 17:30:23.012238 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.012155    2562 scope.go:117] "RemoveContainer" containerID="1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9"
Aug 05 17:30:23.020519 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.020497    2562 scope.go:117] "RemoveContainer" containerID="1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9"
Aug 05 17:30:23.020771 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:30:23.020753    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9\": container with ID starting with 1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9 not found: ID does not exist" containerID="1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9"
Aug 05 17:30:23.020818 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.020779    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9"} err="failed to get container status \"1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9\": rpc error: code = NotFound desc = could not find container \"1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9\": container with ID starting with 1e8adf11811a773ef4ce5390cdec1817d3de42fd34fdfdfef5fc60259a4d7fe9 not found: ID does not exist"
Aug 05 17:30:23.033813 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.033792    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"]
Aug 05 17:30:23.040645 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.040621    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-sches6csn"]
Aug 05 17:30:23.049205 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.049189    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71c935a-5e73-44da-b36e-2464100e34b5-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:23.049276 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:23.049209    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6klf\" (UniqueName: \"kubernetes.io/projected/f71c935a-5e73-44da-b36e-2464100e34b5-kube-api-access-v6klf\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:30:24.103942 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:24.103902    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f71c935a-5e73-44da-b36e-2464100e34b5" path="/var/lib/kubelet/pods/f71c935a-5e73-44da-b36e-2464100e34b5/volumes"
Aug 05 17:30:25.519125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:25.519091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:25.620110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:25.620077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:27.783804 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:27.783766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:34.259244 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:34.259210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:34.359675 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:34.359648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:41.906717 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:41.906683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:41.999927 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:41.999893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:44.976564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:44.976528    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:30:48.703115 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:48.703054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:54.420158 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:54.420128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:30:54.496459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:30:54.496432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:09.744947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:09.744915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:17.280416 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.280382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:17.476395 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.469788    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"]
Aug 05 17:31:17.476395 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.471639    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71c935a-5e73-44da-b36e-2464100e34b5" containerName="main"
Aug 05 17:31:17.476395 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.471665    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71c935a-5e73-44da-b36e-2464100e34b5" containerName="main"
Aug 05 17:31:17.476395 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.472216    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f71c935a-5e73-44da-b36e-2464100e34b5" containerName="main"
Aug 05 17:31:17.482206 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.482178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.482859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.482835    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"]
Aug 05 17:31:17.486381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.486360    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"kube-root-ca.crt\""
Aug 05 17:31:17.486545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.486360    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"openshift-service-ca.crt\""
Aug 05 17:31:17.486632 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.486386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-epp-sa-dockercfg-mttfg\""
Aug 05 17:31:17.486632 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.486414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"default-dockercfg-2v9qm\""
Aug 05 17:31:17.486751 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.486491    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Aug 05 17:31:17.504344 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.504301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:17.573444 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.573366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wn5k\" (UniqueName: \"kubernetes.io/projected/b31fad5c-e75d-42ec-a215-85b5795dfe78-kube-api-access-4wn5k\") pod \"scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.573444 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.573405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b31fad5c-e75d-42ec-a215-85b5795dfe78-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.674406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.674372    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wn5k\" (UniqueName: \"kubernetes.io/projected/b31fad5c-e75d-42ec-a215-85b5795dfe78-kube-api-access-4wn5k\") pod \"scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.674581 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.674420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b31fad5c-e75d-42ec-a215-85b5795dfe78-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.676953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.676927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b31fad5c-e75d-42ec-a215-85b5795dfe78-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.684189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.684167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wn5k\" (UniqueName: \"kubernetes.io/projected/b31fad5c-e75d-42ec-a215-85b5795dfe78-kube-api-access-4wn5k\") pod \"scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.795006 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.794979    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:17.921949 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:17.921916    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"]
Aug 05 17:31:17.924496 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:31:17.924468    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31fad5c_e75d_42ec_a215_85b5795dfe78.slice/crio-5124869672c157b8c72656d4665d17ace0fa24aaed3207c6eb73da9736d5a9ae WatchSource:0}: Error finding container 5124869672c157b8c72656d4665d17ace0fa24aaed3207c6eb73da9736d5a9ae: Status 404 returned error can't find the container with id 5124869672c157b8c72656d4665d17ace0fa24aaed3207c6eb73da9736d5a9ae
Aug 05 17:31:18.269111 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:18.269078    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" event={"ID":"b31fad5c-e75d-42ec-a215-85b5795dfe78","Type":"ContainerStarted","Data":"e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b"}
Aug 05 17:31:18.269271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:18.269129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" event={"ID":"b31fad5c-e75d-42ec-a215-85b5795dfe78","Type":"ContainerStarted","Data":"5124869672c157b8c72656d4665d17ace0fa24aaed3207c6eb73da9736d5a9ae"}
Aug 05 17:31:18.269271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:18.269162    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:18.289570 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:18.289506    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" podStartSLOduration=1.289487611 podStartE2EDuration="1.289487611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:18.288892496 +0000 UTC m=+1136.822076742" watchObservedRunningTime="2026-08-05 17:31:18.289487611 +0000 UTC m=+1136.822671857"
Aug 05 17:31:28.595526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:28.595470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:49.275456 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:49.275378    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:31:50.510575 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:50.510538    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"]
Aug 05 17:31:50.525891 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:50.525855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:51.767533 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:51.767488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:31:59.275665 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:31:59.275628    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" podUID="b31fad5c-e75d-42ec-a215-85b5795dfe78" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:32:05.530258 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.530211    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" podUID="b31fad5c-e75d-42ec-a215-85b5795dfe78" containerName="main" containerID="cri-o://e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b" gracePeriod=45
Aug 05 17:32:05.783965 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.783910    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:32:05.872368 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.872341    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b31fad5c-e75d-42ec-a215-85b5795dfe78-tls-certs\") pod \"b31fad5c-e75d-42ec-a215-85b5795dfe78\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") "
Aug 05 17:32:05.872500 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.872378    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wn5k\" (UniqueName: \"kubernetes.io/projected/b31fad5c-e75d-42ec-a215-85b5795dfe78-kube-api-access-4wn5k\") pod \"b31fad5c-e75d-42ec-a215-85b5795dfe78\" (UID: \"b31fad5c-e75d-42ec-a215-85b5795dfe78\") "
Aug 05 17:32:05.874458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.874417    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31fad5c-e75d-42ec-a215-85b5795dfe78-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b31fad5c-e75d-42ec-a215-85b5795dfe78" (UID: "b31fad5c-e75d-42ec-a215-85b5795dfe78"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:32:05.874564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.874485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31fad5c-e75d-42ec-a215-85b5795dfe78-kube-api-access-4wn5k" (OuterVolumeSpecName: "kube-api-access-4wn5k") pod "b31fad5c-e75d-42ec-a215-85b5795dfe78" (UID: "b31fad5c-e75d-42ec-a215-85b5795dfe78"). InnerVolumeSpecName "kube-api-access-4wn5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:32:05.973655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.973626    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b31fad5c-e75d-42ec-a215-85b5795dfe78-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:32:05.973655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:05.973653    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wn5k\" (UniqueName: \"kubernetes.io/projected/b31fad5c-e75d-42ec-a215-85b5795dfe78-kube-api-access-4wn5k\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:32:06.501548 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.501521    2562 generic.go:356] "Generic (PLEG): container finished" podID="b31fad5c-e75d-42ec-a215-85b5795dfe78" containerID="e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b" exitCode=0
Aug 05 17:32:06.501698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.501565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"
Aug 05 17:32:06.501698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.501583    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" event={"ID":"b31fad5c-e75d-42ec-a215-85b5795dfe78","Type":"ContainerDied","Data":"e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b"}
Aug 05 17:32:06.501698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.501625    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl" event={"ID":"b31fad5c-e75d-42ec-a215-85b5795dfe78","Type":"ContainerDied","Data":"5124869672c157b8c72656d4665d17ace0fa24aaed3207c6eb73da9736d5a9ae"}
Aug 05 17:32:06.501698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.501646    2562 scope.go:117] "RemoveContainer" containerID="e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b"
Aug 05 17:32:06.509876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.509859    2562 scope.go:117] "RemoveContainer" containerID="e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b"
Aug 05 17:32:06.510113 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:32:06.510094    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b\": container with ID starting with e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b not found: ID does not exist" containerID="e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b"
Aug 05 17:32:06.510159 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.510121    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b"} err="failed to get container status \"e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b\": rpc error: code = NotFound desc = could not find container \"e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b\": container with ID starting with e83caf028f69a14db477a2e124a896102921f07282f62c1b97a79fb9756c532b not found: ID does not exist"
Aug 05 17:32:06.518898 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.518877    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"]
Aug 05 17:32:06.522751 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:06.522732    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-6bbd5l6krl"]
Aug 05 17:32:08.101992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:08.101955    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b31fad5c-e75d-42ec-a215-85b5795dfe78" path="/var/lib/kubelet/pods/b31fad5c-e75d-42ec-a215-85b5795dfe78/volumes"
Aug 05 17:32:10.526939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:10.526909    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:10.627281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:10.627248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:16.950010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:16.949971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:18.243453 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.243423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:18.477685 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.477651    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"]
Aug 05 17:32:18.478045 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.478032    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b31fad5c-e75d-42ec-a215-85b5795dfe78" containerName="main"
Aug 05 17:32:18.478089 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.478046    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b31fad5c-e75d-42ec-a215-85b5795dfe78" containerName="main"
Aug 05 17:32:18.478127 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.478107    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b31fad5c-e75d-42ec-a215-85b5795dfe78" containerName="main"
Aug 05 17:32:18.482905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.482879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.486027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.486005    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\""
Aug 05 17:32:18.487851 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.487644    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-7xwp5\""
Aug 05 17:32:18.488019 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.487995    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-lpfpm\""
Aug 05 17:32:18.488094 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.488063    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Aug 05 17:32:18.488256 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.488238    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\""
Aug 05 17:32:18.489792 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.489774    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"]
Aug 05 17:32:18.510254 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.510200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:18.569201 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.569172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.569292 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.569202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brt72\" (UniqueName: \"kubernetes.io/projected/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-kube-api-access-brt72\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.669627 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.669599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.669737 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.669631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brt72\" (UniqueName: \"kubernetes.io/projected/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-kube-api-access-brt72\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.672129 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.672109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.677689 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.677670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brt72\" (UniqueName: \"kubernetes.io/projected/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-kube-api-access-brt72\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.796164 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.796103    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:18.926819 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:18.926797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"]
Aug 05 17:32:18.929193 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:32:18.929166    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a1868e_1b3f_4ad1_aa6a_bbef8c14e6f2.slice/crio-9cac32427468ed8fe7ff982415d87463b622f08f7fa7990bf94d6dbc98f175eb WatchSource:0}: Error finding container 9cac32427468ed8fe7ff982415d87463b622f08f7fa7990bf94d6dbc98f175eb: Status 404 returned error can't find the container with id 9cac32427468ed8fe7ff982415d87463b622f08f7fa7990bf94d6dbc98f175eb
Aug 05 17:32:19.563913 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:19.563865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" event={"ID":"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2","Type":"ContainerStarted","Data":"85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01"}
Aug 05 17:32:19.564327 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:19.563927    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" event={"ID":"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2","Type":"ContainerStarted","Data":"9cac32427468ed8fe7ff982415d87463b622f08f7fa7990bf94d6dbc98f175eb"}
Aug 05 17:32:19.564327 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:19.563955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:19.582702 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:19.582657    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" podStartSLOduration=1.582638905 podStartE2EDuration="1.582638905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:32:19.58140283 +0000 UTC m=+1198.114587076" watchObservedRunningTime="2026-08-05 17:32:19.582638905 +0000 UTC m=+1198.115823150"
Aug 05 17:32:21.927189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:21.927163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:32:21.930592 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:21.930568    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:32:21.932186 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:21.932168    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:32:21.934984 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:21.934964    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:32:23.085398 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:23.085372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:23.192020 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:23.191993    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:34.803816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:34.803785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:36.608559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:36.608526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:36.623995 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:36.623968    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"]
Aug 05 17:32:36.675257 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:36.675223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:38.473392 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:38.473362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:38.529790 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:38.529761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:44.975473 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:44.975427    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" podUID="017b6c68-1344-4d9c-891a-99daeebc10d8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:32:50.569901 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:50.569865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:32:51.647771 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:51.647730    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" podUID="017b6c68-1344-4d9c-891a-99daeebc10d8" containerName="main" containerID="cri-o://fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f" gracePeriod=45
Aug 05 17:32:51.827903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:51.827867    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"]
Aug 05 17:32:51.830902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:51.830844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:51.927233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:51.927200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:51.936608 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:51.936584    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:32:52.050882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.050847    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6c9x\" (UniqueName: \"kubernetes.io/projected/017b6c68-1344-4d9c-891a-99daeebc10d8-kube-api-access-k6c9x\") pod \"017b6c68-1344-4d9c-891a-99daeebc10d8\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") "
Aug 05 17:32:52.051076 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.050944    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017b6c68-1344-4d9c-891a-99daeebc10d8-tls-certs\") pod \"017b6c68-1344-4d9c-891a-99daeebc10d8\" (UID: \"017b6c68-1344-4d9c-891a-99daeebc10d8\") "
Aug 05 17:32:52.053135 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.053108    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/017b6c68-1344-4d9c-891a-99daeebc10d8-kube-api-access-k6c9x" (OuterVolumeSpecName: "kube-api-access-k6c9x") pod "017b6c68-1344-4d9c-891a-99daeebc10d8" (UID: "017b6c68-1344-4d9c-891a-99daeebc10d8"). InnerVolumeSpecName "kube-api-access-k6c9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:32:52.053224 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.053110    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/017b6c68-1344-4d9c-891a-99daeebc10d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "017b6c68-1344-4d9c-891a-99daeebc10d8" (UID: "017b6c68-1344-4d9c-891a-99daeebc10d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:32:52.152207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.152174    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017b6c68-1344-4d9c-891a-99daeebc10d8-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:32:52.152207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.152202    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6c9x\" (UniqueName: \"kubernetes.io/projected/017b6c68-1344-4d9c-891a-99daeebc10d8-kube-api-access-k6c9x\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:32:52.723723 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.723685    2562 generic.go:356] "Generic (PLEG): container finished" podID="017b6c68-1344-4d9c-891a-99daeebc10d8" containerID="fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f" exitCode=0
Aug 05 17:32:52.724229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.723735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" event={"ID":"017b6c68-1344-4d9c-891a-99daeebc10d8","Type":"ContainerDied","Data":"fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f"}
Aug 05 17:32:52.724229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.723754    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"
Aug 05 17:32:52.724229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.723799    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r" event={"ID":"017b6c68-1344-4d9c-891a-99daeebc10d8","Type":"ContainerDied","Data":"331cea3eb392218e6c323ff4e1b95281128dafec77aff14574084d5fb7b94fc7"}
Aug 05 17:32:52.724229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.723821    2562 scope.go:117] "RemoveContainer" containerID="fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f"
Aug 05 17:32:52.732616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.732599    2562 scope.go:117] "RemoveContainer" containerID="fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f"
Aug 05 17:32:52.732857 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:32:52.732835    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f\": container with ID starting with fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f not found: ID does not exist" containerID="fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f"
Aug 05 17:32:52.732905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.732866    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f"} err="failed to get container status \"fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f\": rpc error: code = NotFound desc = could not find container \"fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f\": container with ID starting with fbc6a4fdfe27b9e7597f2586997aa3bd675acedbd93c9d0cc007407544bcac1f not found: ID does not exist"
Aug 05 17:32:52.741611 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.741584    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"]
Aug 05 17:32:52.747287 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:52.747266    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-m766r"]
Aug 05 17:32:54.102829 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:54.102795    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="017b6c68-1344-4d9c-891a-99daeebc10d8" path="/var/lib/kubelet/pods/017b6c68-1344-4d9c-891a-99daeebc10d8/volumes"
Aug 05 17:32:56.482286 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:56.482256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:32:56.586993 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:32:56.586956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:00.570042 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:00.570006    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" podUID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:33:02.961562 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:02.961529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:06.204217 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.204186    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:06.452952 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.452918    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"]
Aug 05 17:33:06.453283 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.453271    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="017b6c68-1344-4d9c-891a-99daeebc10d8" containerName="main"
Aug 05 17:33:06.453347 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.453284    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="017b6c68-1344-4d9c-891a-99daeebc10d8" containerName="main"
Aug 05 17:33:06.453389 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.453382    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="017b6c68-1344-4d9c-891a-99daeebc10d8" containerName="main"
Aug 05 17:33:06.457919 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.457867    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.460849 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.460825    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\""
Aug 05 17:33:06.460849 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.460836    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-epp-sa-dockercfg-dnvdk\""
Aug 05 17:33:06.461017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.460831    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Aug 05 17:33:06.461017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.460830    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-djpld\""
Aug 05 17:33:06.461017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.460932    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\""
Aug 05 17:33:06.465613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.465590    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"]
Aug 05 17:33:06.479722 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.479690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43d8787-fea9-4c2e-afdd-8e528bd04db7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.479863 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.479760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6nf2\" (UniqueName: \"kubernetes.io/projected/d43d8787-fea9-4c2e-afdd-8e528bd04db7-kube-api-access-k6nf2\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.480382 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.480361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:06.580679 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.580644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43d8787-fea9-4c2e-afdd-8e528bd04db7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.580878 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.580691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nf2\" (UniqueName: \"kubernetes.io/projected/d43d8787-fea9-4c2e-afdd-8e528bd04db7-kube-api-access-k6nf2\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.583136 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.583112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43d8787-fea9-4c2e-afdd-8e528bd04db7-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.589600 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.589574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6nf2\" (UniqueName: \"kubernetes.io/projected/d43d8787-fea9-4c2e-afdd-8e528bd04db7-kube-api-access-k6nf2\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.769051 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.768963    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:06.859625 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.859562    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" podUID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" containerName="main" containerID="cri-o://85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01" gracePeriod=45
Aug 05 17:33:06.903162 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:06.903134    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"]
Aug 05 17:33:06.922336 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:33:06.922280    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43d8787_fea9_4c2e_afdd_8e528bd04db7.slice/crio-5c8d3b4431ffdaf9019601e12d93cc2e0c0d3d6989b44dd09d9e877d2362d4fb WatchSource:0}: Error finding container 5c8d3b4431ffdaf9019601e12d93cc2e0c0d3d6989b44dd09d9e877d2362d4fb: Status 404 returned error can't find the container with id 5c8d3b4431ffdaf9019601e12d93cc2e0c0d3d6989b44dd09d9e877d2362d4fb
Aug 05 17:33:07.121707 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.121679    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:33:07.187588 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.187553    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brt72\" (UniqueName: \"kubernetes.io/projected/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-kube-api-access-brt72\") pod \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") "
Aug 05 17:33:07.187772 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.187690    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-tls-certs\") pod \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\" (UID: \"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2\") "
Aug 05 17:33:07.189895 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.189861    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" (UID: "37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:33:07.190018 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.189904    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-kube-api-access-brt72" (OuterVolumeSpecName: "kube-api-access-brt72") pod "37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" (UID: "37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2"). InnerVolumeSpecName "kube-api-access-brt72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:33:07.288951 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.288870    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:33:07.288951 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.288899    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brt72\" (UniqueName: \"kubernetes.io/projected/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2-kube-api-access-brt72\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:33:07.797787 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.797742    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" event={"ID":"d43d8787-fea9-4c2e-afdd-8e528bd04db7","Type":"ContainerStarted","Data":"e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412"}
Aug 05 17:33:07.797994 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.797806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" event={"ID":"d43d8787-fea9-4c2e-afdd-8e528bd04db7","Type":"ContainerStarted","Data":"5c8d3b4431ffdaf9019601e12d93cc2e0c0d3d6989b44dd09d9e877d2362d4fb"}
Aug 05 17:33:07.797994 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.797833    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:07.798976 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.798953    2562 generic.go:356] "Generic (PLEG): container finished" podID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" containerID="85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01" exitCode=0
Aug 05 17:33:07.799067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.798983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" event={"ID":"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2","Type":"ContainerDied","Data":"85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01"}
Aug 05 17:33:07.799067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.798990    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"
Aug 05 17:33:07.799067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.799012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w" event={"ID":"37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2","Type":"ContainerDied","Data":"9cac32427468ed8fe7ff982415d87463b622f08f7fa7990bf94d6dbc98f175eb"}
Aug 05 17:33:07.799067 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.799032    2562 scope.go:117] "RemoveContainer" containerID="85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01"
Aug 05 17:33:07.807698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.807679    2562 scope.go:117] "RemoveContainer" containerID="85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01"
Aug 05 17:33:07.807966 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:33:07.807948    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01\": container with ID starting with 85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01 not found: ID does not exist" containerID="85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01"
Aug 05 17:33:07.808008 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.807975    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01"} err="failed to get container status \"85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01\": rpc error: code = NotFound desc = could not find container \"85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01\": container with ID starting with 85be39b08d2bc3d9c5b59a3986656870e069eb3e849ec688155b43e22ea26f01 not found: ID does not exist"
Aug 05 17:33:07.818977 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.818929    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" podStartSLOduration=1.818916362 podStartE2EDuration="1.818916362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:07.816184698 +0000 UTC m=+1246.349368962" watchObservedRunningTime="2026-08-05 17:33:07.818916362 +0000 UTC m=+1246.352100606"
Aug 05 17:33:07.830745 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.830705    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"]
Aug 05 17:33:07.839228 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:07.839201    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche2f26w"]
Aug 05 17:33:08.102944 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:08.102911    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" path="/var/lib/kubelet/pods/37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2/volumes"
Aug 05 17:33:09.011695 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:09.011664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:09.112235 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:09.112206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:11.666621 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:11.666585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:11.758432 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:11.758401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:17.137810 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:17.137739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:18.361336 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:18.361287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:22.640482 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.640454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:22.816085 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.816053    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"]
Aug 05 17:33:22.816529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.816510    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" containerName="main"
Aug 05 17:33:22.816636 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.816531    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" containerName="main"
Aug 05 17:33:22.816694 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.816638    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="37a1868e-1b3f-4ad1-aa6a-bbef8c14e6f2" containerName="main"
Aug 05 17:33:22.820974 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.820952    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:22.824477 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.824455    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\""
Aug 05 17:33:22.824598 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.824487    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\""
Aug 05 17:33:22.824598 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.824498    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-6bdnj\""
Aug 05 17:33:22.824598 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.824507    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-9cvxq\""
Aug 05 17:33:22.824744 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.824606    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Aug 05 17:33:22.828146 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.828126    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"]
Aug 05 17:33:22.844227 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.844196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:22.914890 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.914814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8ql5\" (UniqueName: \"kubernetes.io/projected/3259a2f7-fa13-43b1-853a-e98403ced0bc-kube-api-access-s8ql5\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:22.914890 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:22.914873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3259a2f7-fa13-43b1-853a-e98403ced0bc-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.015846 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.015813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8ql5\" (UniqueName: \"kubernetes.io/projected/3259a2f7-fa13-43b1-853a-e98403ced0bc-kube-api-access-s8ql5\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.015979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.015870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3259a2f7-fa13-43b1-853a-e98403ced0bc-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.018411 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.018388    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3259a2f7-fa13-43b1-853a-e98403ced0bc-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.024681 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.024655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8ql5\" (UniqueName: \"kubernetes.io/projected/3259a2f7-fa13-43b1-853a-e98403ced0bc-kube-api-access-s8ql5\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.132751 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.132719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.269407 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.269203    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"]
Aug 05 17:33:23.276771 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:33:23.271774    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3259a2f7_fa13_43b1_853a_e98403ced0bc.slice/crio-3d6f7680ed40ffe3b49413cdd181bb91c693df3aaf5b1c96ad52844704638120 WatchSource:0}: Error finding container 3d6f7680ed40ffe3b49413cdd181bb91c693df3aaf5b1c96ad52844704638120: Status 404 returned error can't find the container with id 3d6f7680ed40ffe3b49413cdd181bb91c693df3aaf5b1c96ad52844704638120
Aug 05 17:33:23.876497 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.876452    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" event={"ID":"3259a2f7-fa13-43b1-853a-e98403ced0bc","Type":"ContainerStarted","Data":"5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721"}
Aug 05 17:33:23.877065 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.876558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" event={"ID":"3259a2f7-fa13-43b1-853a-e98403ced0bc","Type":"ContainerStarted","Data":"3d6f7680ed40ffe3b49413cdd181bb91c693df3aaf5b1c96ad52844704638120"}
Aug 05 17:33:23.877065 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.876723    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:23.896465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:23.896411    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" podStartSLOduration=1.896395938 podStartE2EDuration="1.896395938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:23.894953115 +0000 UTC m=+1262.428137373" watchObservedRunningTime="2026-08-05 17:33:23.896395938 +0000 UTC m=+1262.429580183"
Aug 05 17:33:25.067096 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:25.067066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:25.167431 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:25.167376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:38.095546 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:38.095505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:38.806488 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:38.806449    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:33:54.882979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:54.882948    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:33:56.524784 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:56.524755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:33:56.530686 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:56.530651    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"]
Aug 05 17:33:56.617480 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:33:56.617451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:04.883265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:04.883220    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" podUID="3259a2f7-fa13-43b1-853a-e98403ced0bc" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:34:11.549478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.549428    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" podUID="3259a2f7-fa13-43b1-853a-e98403ced0bc" containerName="main" containerID="cri-o://5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721" gracePeriod=45
Aug 05 17:34:11.788679 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.788655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:34:11.825248 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.825175    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8ql5\" (UniqueName: \"kubernetes.io/projected/3259a2f7-fa13-43b1-853a-e98403ced0bc-kube-api-access-s8ql5\") pod \"3259a2f7-fa13-43b1-853a-e98403ced0bc\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") "
Aug 05 17:34:11.825386 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.825273    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3259a2f7-fa13-43b1-853a-e98403ced0bc-tls-certs\") pod \"3259a2f7-fa13-43b1-853a-e98403ced0bc\" (UID: \"3259a2f7-fa13-43b1-853a-e98403ced0bc\") "
Aug 05 17:34:11.827345 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.827293    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3259a2f7-fa13-43b1-853a-e98403ced0bc-kube-api-access-s8ql5" (OuterVolumeSpecName: "kube-api-access-s8ql5") pod "3259a2f7-fa13-43b1-853a-e98403ced0bc" (UID: "3259a2f7-fa13-43b1-853a-e98403ced0bc"). InnerVolumeSpecName "kube-api-access-s8ql5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:34:11.827345 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.827330    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3259a2f7-fa13-43b1-853a-e98403ced0bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3259a2f7-fa13-43b1-853a-e98403ced0bc" (UID: "3259a2f7-fa13-43b1-853a-e98403ced0bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:34:11.926748 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.926714    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3259a2f7-fa13-43b1-853a-e98403ced0bc-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:34:11.926748 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:11.926742    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8ql5\" (UniqueName: \"kubernetes.io/projected/3259a2f7-fa13-43b1-853a-e98403ced0bc-kube-api-access-s8ql5\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:34:12.105038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.104951    2562 generic.go:356] "Generic (PLEG): container finished" podID="3259a2f7-fa13-43b1-853a-e98403ced0bc" containerID="5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721" exitCode=0
Aug 05 17:34:12.105038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.104986    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"
Aug 05 17:34:12.105038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.105027    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" event={"ID":"3259a2f7-fa13-43b1-853a-e98403ced0bc","Type":"ContainerDied","Data":"5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721"}
Aug 05 17:34:12.105328 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.105069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq" event={"ID":"3259a2f7-fa13-43b1-853a-e98403ced0bc","Type":"ContainerDied","Data":"3d6f7680ed40ffe3b49413cdd181bb91c693df3aaf5b1c96ad52844704638120"}
Aug 05 17:34:12.105328 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.105092    2562 scope.go:117] "RemoveContainer" containerID="5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721"
Aug 05 17:34:12.123428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.123407    2562 scope.go:117] "RemoveContainer" containerID="5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721"
Aug 05 17:34:12.123693 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:34:12.123670    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721\": container with ID starting with 5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721 not found: ID does not exist" containerID="5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721"
Aug 05 17:34:12.123769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.123704    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721"} err="failed to get container status \"5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721\": rpc error: code = NotFound desc = could not find container \"5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721\": container with ID starting with 5f79437fdf60c07709dd82d5cdea134f7ba5d1a37a6716ff6e062913057ad721 not found: ID does not exist"
Aug 05 17:34:12.136135 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.136106    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"]
Aug 05 17:34:12.140038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:12.140015    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-56587bgwnq"]
Aug 05 17:34:14.102196 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:14.102162    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3259a2f7-fa13-43b1-853a-e98403ced0bc" path="/var/lib/kubelet/pods/3259a2f7-fa13-43b1-853a-e98403ced0bc/volumes"
Aug 05 17:34:16.525743 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:16.525711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:16.622470 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:16.622440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:19.814476 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:19.814445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:20.831532 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:20.831500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:21.002215 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.002181    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"]
Aug 05 17:34:21.002565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.002552    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3259a2f7-fa13-43b1-853a-e98403ced0bc" containerName="main"
Aug 05 17:34:21.002617 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.002566    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3259a2f7-fa13-43b1-853a-e98403ced0bc" containerName="main"
Aug 05 17:34:21.002650 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.002631    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3259a2f7-fa13-43b1-853a-e98403ced0bc" containerName="main"
Aug 05 17:34:21.007041 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.007021    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.010695 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.010666    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-6sr77\""
Aug 05 17:34:21.010816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.010698    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-ffggt\""
Aug 05 17:34:21.010816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.010763    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Aug 05 17:34:21.010816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.010773    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Aug 05 17:34:21.010816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.010799    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Aug 05 17:34:21.016072 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.016052    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"]
Aug 05 17:34:21.026648 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.026626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:21.102813 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.102757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/357ae95a-dcd2-44bc-87bd-a42a53734196-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.102922 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.102822    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sxft\" (UniqueName: \"kubernetes.io/projected/357ae95a-dcd2-44bc-87bd-a42a53734196-kube-api-access-5sxft\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.203396 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.203364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/357ae95a-dcd2-44bc-87bd-a42a53734196-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.203566 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.203429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sxft\" (UniqueName: \"kubernetes.io/projected/357ae95a-dcd2-44bc-87bd-a42a53734196-kube-api-access-5sxft\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.205880 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.205848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/357ae95a-dcd2-44bc-87bd-a42a53734196-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.211694 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.211673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sxft\" (UniqueName: \"kubernetes.io/projected/357ae95a-dcd2-44bc-87bd-a42a53734196-kube-api-access-5sxft\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.319157 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.319119    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:21.451027 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:34:21.450995    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357ae95a_dcd2_44bc_87bd_a42a53734196.slice/crio-3ac55dc47238cf1c49660a255506db52a0a892312192964798b665727eedb38a WatchSource:0}: Error finding container 3ac55dc47238cf1c49660a255506db52a0a892312192964798b665727eedb38a: Status 404 returned error can't find the container with id 3ac55dc47238cf1c49660a255506db52a0a892312192964798b665727eedb38a
Aug 05 17:34:21.451622 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:21.451561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"]
Aug 05 17:34:22.150126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:22.150092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" event={"ID":"357ae95a-dcd2-44bc-87bd-a42a53734196","Type":"ContainerStarted","Data":"f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"}
Aug 05 17:34:22.150500 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:22.150144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" event={"ID":"357ae95a-dcd2-44bc-87bd-a42a53734196","Type":"ContainerStarted","Data":"3ac55dc47238cf1c49660a255506db52a0a892312192964798b665727eedb38a"}
Aug 05 17:34:22.150500 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:22.150167    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:22.168942 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:22.168897    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" podStartSLOduration=2.168883376 podStartE2EDuration="2.168883376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:34:22.166960313 +0000 UTC m=+1320.700144559" watchObservedRunningTime="2026-08-05 17:34:22.168883376 +0000 UTC m=+1320.702067620"
Aug 05 17:34:23.685559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:23.685523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:29.658110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:29.658083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:29.757928 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:29.757900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:30.717134 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:30.717097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:53.156639 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:53.156556    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:34:54.919423 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:54.919392    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"]
Aug 05 17:34:54.932023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:54.931996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:34:54.986561 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:34:54.986527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:03.156110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:03.156071    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:35:05.719650 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.719619    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" message="" err="command '/bin/sleep 15' exited with 137: "
Aug 05 17:35:05.719991 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:35:05.719652    2562 kuberuntime_container.go:755] "PreStop hook failed" err="command '/bin/sleep 15' exited with 137: " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerName="main" containerID="cri-o://f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"
Aug 05 17:35:05.719991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.719715    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerName="main" containerID="cri-o://f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2" gracePeriod=50
Aug 05 17:35:05.857019 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.856996    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp_357ae95a-dcd2-44bc-87bd-a42a53734196/main/0.log"
Aug 05 17:35:05.859133 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.859118    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:35:05.976023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.975947    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sxft\" (UniqueName: \"kubernetes.io/projected/357ae95a-dcd2-44bc-87bd-a42a53734196-kube-api-access-5sxft\") pod \"357ae95a-dcd2-44bc-87bd-a42a53734196\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") "
Aug 05 17:35:05.976185 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.976049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/357ae95a-dcd2-44bc-87bd-a42a53734196-tls-certs\") pod \"357ae95a-dcd2-44bc-87bd-a42a53734196\" (UID: \"357ae95a-dcd2-44bc-87bd-a42a53734196\") "
Aug 05 17:35:05.978118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.978092    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/357ae95a-dcd2-44bc-87bd-a42a53734196-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "357ae95a-dcd2-44bc-87bd-a42a53734196" (UID: "357ae95a-dcd2-44bc-87bd-a42a53734196"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:05.978118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:05.978104    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/357ae95a-dcd2-44bc-87bd-a42a53734196-kube-api-access-5sxft" (OuterVolumeSpecName: "kube-api-access-5sxft") pod "357ae95a-dcd2-44bc-87bd-a42a53734196" (UID: "357ae95a-dcd2-44bc-87bd-a42a53734196"). InnerVolumeSpecName "kube-api-access-5sxft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:06.076813 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.076774    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/357ae95a-dcd2-44bc-87bd-a42a53734196-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:35:06.076813 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.076811    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sxft\" (UniqueName: \"kubernetes.io/projected/357ae95a-dcd2-44bc-87bd-a42a53734196-kube-api-access-5sxft\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:35:06.364643 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.364575    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp_357ae95a-dcd2-44bc-87bd-a42a53734196/main/0.log"
Aug 05 17:35:06.364643 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.364613    2562 generic.go:356] "Generic (PLEG): container finished" podID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerID="f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2" exitCode=1
Aug 05 17:35:06.364814 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.364662    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"
Aug 05 17:35:06.364814 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.364710    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" event={"ID":"357ae95a-dcd2-44bc-87bd-a42a53734196","Type":"ContainerDied","Data":"f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"}
Aug 05 17:35:06.364814 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.364759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp" event={"ID":"357ae95a-dcd2-44bc-87bd-a42a53734196","Type":"ContainerDied","Data":"3ac55dc47238cf1c49660a255506db52a0a892312192964798b665727eedb38a"}
Aug 05 17:35:06.364814 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.364783    2562 scope.go:117] "RemoveContainer" containerID="f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"
Aug 05 17:35:06.372910 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.372892    2562 scope.go:117] "RemoveContainer" containerID="f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"
Aug 05 17:35:06.373179 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:35:06.373158    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2\": container with ID starting with f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2 not found: ID does not exist" containerID="f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"
Aug 05 17:35:06.373229 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.373187    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2"} err="failed to get container status \"f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2\": rpc error: code = NotFound desc = could not find container \"f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2\": container with ID starting with f624a294a4c972f9f736b279c429cfc0126fc4ccc2c0d9a029fe7d78e15b15d2 not found: ID does not exist"
Aug 05 17:35:06.384978 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.384956    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"]
Aug 05 17:35:06.389088 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:06.389069    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-66cb9cfnkmwp"]
Aug 05 17:35:07.963033 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:07.963000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:07.970584 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:07.970562    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"]
Aug 05 17:35:08.020440 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:08.020410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:08.102870 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:08.102835    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" path="/var/lib/kubelet/pods/357ae95a-dcd2-44bc-87bd-a42a53734196/volumes"
Aug 05 17:35:08.807220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:08.807183    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:35:14.963663 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:14.963627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:15.057353 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:15.057308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:18.807218 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:18.807177    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:35:20.910882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:20.910848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:21.114095 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.114062    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"]
Aug 05 17:35:21.114544 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.114525    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerName="main"
Aug 05 17:35:21.114544 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.114545    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerName="main"
Aug 05 17:35:21.114698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.114653    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="357ae95a-dcd2-44bc-87bd-a42a53734196" containerName="main"
Aug 05 17:35:21.118862 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.118840    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.122087 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.122066    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"kube-root-ca.crt\""
Aug 05 17:35:21.123237 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.123220    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Aug 05 17:35:21.123376 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.123263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"default-dockercfg-j4crz\""
Aug 05 17:35:21.123376 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.123307    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-epp-sa-dockercfg-xv28b\""
Aug 05 17:35:21.123518 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.123492    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"openshift-service-ca.crt\""
Aug 05 17:35:21.135018 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.134994    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"]
Aug 05 17:35:21.145576 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.145556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:21.209319 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.209287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.209505 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.209369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea491680-1b9d-4458-a9b9-647e8978da79-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.209505 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.209388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8pfq\" (UniqueName: \"kubernetes.io/projected/ea491680-1b9d-4458-a9b9-647e8978da79-kube-api-access-l8pfq\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.209505 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.209431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.310406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.310342    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.310580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.310470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.310580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.310516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea491680-1b9d-4458-a9b9-647e8978da79-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.310580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.310538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8pfq\" (UniqueName: \"kubernetes.io/projected/ea491680-1b9d-4458-a9b9-647e8978da79-kube-api-access-l8pfq\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.310783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.310757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.310838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.310789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.312897 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.312877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea491680-1b9d-4458-a9b9-647e8978da79-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.319280 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.319255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8pfq\" (UniqueName: \"kubernetes.io/projected/ea491680-1b9d-4458-a9b9-647e8978da79-kube-api-access-l8pfq\") pod \"scheduler-custom-template-test-kserve-router-scheduler-7957tcm9\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.429769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.429733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:21.581257 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:21.581231    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"]
Aug 05 17:35:21.583892 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:35:21.583863    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea491680_1b9d_4458_a9b9_647e8978da79.slice/crio-e82bd992b71651a4b77198c26a603ab8f20a380f876df08341de55d89bbd95ca WatchSource:0}: Error finding container e82bd992b71651a4b77198c26a603ab8f20a380f876df08341de55d89bbd95ca: Status 404 returned error can't find the container with id e82bd992b71651a4b77198c26a603ab8f20a380f876df08341de55d89bbd95ca
Aug 05 17:35:22.440233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:22.440155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:22.440610 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:22.440243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" event={"ID":"ea491680-1b9d-4458-a9b9-647e8978da79","Type":"ContainerStarted","Data":"415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360"}
Aug 05 17:35:22.440610 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:22.440284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" event={"ID":"ea491680-1b9d-4458-a9b9-647e8978da79","Type":"ContainerStarted","Data":"e82bd992b71651a4b77198c26a603ab8f20a380f876df08341de55d89bbd95ca"}
Aug 05 17:35:22.440610 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:22.440339    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:35:22.464547 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:22.464501    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" podStartSLOduration=1.4644864260000001 podStartE2EDuration="1.464486426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:35:22.462828478 +0000 UTC m=+1380.996012750" watchObservedRunningTime="2026-08-05 17:35:22.464486426 +0000 UTC m=+1380.997670669"
Aug 05 17:35:22.995336 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:22.995248    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerName="main" containerID="cri-o://e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412" gracePeriod=45
Aug 05 17:35:23.239870 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.239845    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:35:23.328119 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.328041    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43d8787-fea9-4c2e-afdd-8e528bd04db7-tls-certs\") pod \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") "
Aug 05 17:35:23.328263 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.328137    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6nf2\" (UniqueName: \"kubernetes.io/projected/d43d8787-fea9-4c2e-afdd-8e528bd04db7-kube-api-access-k6nf2\") pod \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\" (UID: \"d43d8787-fea9-4c2e-afdd-8e528bd04db7\") "
Aug 05 17:35:23.330356 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.330303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d43d8787-fea9-4c2e-afdd-8e528bd04db7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d43d8787-fea9-4c2e-afdd-8e528bd04db7" (UID: "d43d8787-fea9-4c2e-afdd-8e528bd04db7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:35:23.330474 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.330362    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43d8787-fea9-4c2e-afdd-8e528bd04db7-kube-api-access-k6nf2" (OuterVolumeSpecName: "kube-api-access-k6nf2") pod "d43d8787-fea9-4c2e-afdd-8e528bd04db7" (UID: "d43d8787-fea9-4c2e-afdd-8e528bd04db7"). InnerVolumeSpecName "kube-api-access-k6nf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:35:23.428924 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.428890    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43d8787-fea9-4c2e-afdd-8e528bd04db7-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:35:23.428924 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.428921    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6nf2\" (UniqueName: \"kubernetes.io/projected/d43d8787-fea9-4c2e-afdd-8e528bd04db7-kube-api-access-k6nf2\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:35:23.446050 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.446017    2562 generic.go:356] "Generic (PLEG): container finished" podID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerID="e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412" exitCode=0
Aug 05 17:35:23.446464 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.446063    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"
Aug 05 17:35:23.446464 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.446102    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" event={"ID":"d43d8787-fea9-4c2e-afdd-8e528bd04db7","Type":"ContainerDied","Data":"e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412"}
Aug 05 17:35:23.446464 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.446177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d" event={"ID":"d43d8787-fea9-4c2e-afdd-8e528bd04db7","Type":"ContainerDied","Data":"5c8d3b4431ffdaf9019601e12d93cc2e0c0d3d6989b44dd09d9e877d2362d4fb"}
Aug 05 17:35:23.446464 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.446200    2562 scope.go:117] "RemoveContainer" containerID="e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412"
Aug 05 17:35:23.455352 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.455331    2562 scope.go:117] "RemoveContainer" containerID="e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412"
Aug 05 17:35:23.455601 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:35:23.455578    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412\": container with ID starting with e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412 not found: ID does not exist" containerID="e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412"
Aug 05 17:35:23.455648 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.455609    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412"} err="failed to get container status \"e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412\": rpc error: code = NotFound desc = could not find container \"e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412\": container with ID starting with e457e8a9f11b58b8d2a67d946192a04f7d61b478a173a057f44aeb1bddd06412 not found: ID does not exist"
Aug 05 17:35:23.468759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.468732    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"]
Aug 05 17:35:23.472457 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:23.472435    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcpxg4d"]
Aug 05 17:35:24.102743 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:24.102708    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" path="/var/lib/kubelet/pods/d43d8787-fea9-4c2e-afdd-8e528bd04db7/volumes"
Aug 05 17:35:28.022129 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:28.022093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:28.146184 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:28.146152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:29.336602 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:29.336573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:29.438038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:29.438011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:34.310489 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:34.310457    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:38.642896 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:38.642863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:38.745557 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:38.745526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:40.272683 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:40.272651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:40.373804 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:40.373771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:47.353049 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:47.353022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:35:53.447477 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:35:53.447446    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:36:14.719902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:14.719820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:23.351116 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.351079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:23.594253 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.594218    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"]
Aug 05 17:36:23.594634 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.594619    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerName="main"
Aug 05 17:36:23.594681 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.594637    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerName="main"
Aug 05 17:36:23.594720 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.594695    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d43d8787-fea9-4c2e-afdd-8e528bd04db7" containerName="main"
Aug 05 17:36:23.597765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.597746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.602644 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.602581    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"kube-root-ca.crt\""
Aug 05 17:36:23.603809 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.603776    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-kserve-self-signed-certs\""
Aug 05 17:36:23.603809 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.603792    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"default-dockercfg-fg72r\""
Aug 05 17:36:23.603979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.603795    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"openshift-service-ca.crt\""
Aug 05 17:36:23.603979 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.603836    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-epp-sa-dockercfg-5b8z9\""
Aug 05 17:36:23.610652 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.610626    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"]
Aug 05 17:36:23.618246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.618220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:23.744591 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.744556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd246b1-c940-4d9e-bad1-242827aa9b73-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.744767 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.744595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvgs9\" (UniqueName: \"kubernetes.io/projected/5bd246b1-c940-4d9e-bad1-242827aa9b73-kube-api-access-mvgs9\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.845784 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.845749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd246b1-c940-4d9e-bad1-242827aa9b73-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.845784 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.845790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgs9\" (UniqueName: \"kubernetes.io/projected/5bd246b1-c940-4d9e-bad1-242827aa9b73-kube-api-access-mvgs9\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.848225 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.848199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd246b1-c940-4d9e-bad1-242827aa9b73-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.856442 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.856386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvgs9\" (UniqueName: \"kubernetes.io/projected/5bd246b1-c940-4d9e-bad1-242827aa9b73-kube-api-access-mvgs9\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:23.907997 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:23.907963    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:24.043336 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:24.043280    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"]
Aug 05 17:36:24.046238 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:36:24.046202    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd246b1_c940_4d9e_bad1_242827aa9b73.slice/crio-ac53149471c827b00e5609786f79d930abfe3fb6fdabdba7031a7c4cdff35940 WatchSource:0}: Error finding container ac53149471c827b00e5609786f79d930abfe3fb6fdabdba7031a7c4cdff35940: Status 404 returned error can't find the container with id ac53149471c827b00e5609786f79d930abfe3fb6fdabdba7031a7c4cdff35940
Aug 05 17:36:24.735855 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:24.735817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" event={"ID":"5bd246b1-c940-4d9e-bad1-242827aa9b73","Type":"ContainerStarted","Data":"b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0"}
Aug 05 17:36:24.736281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:24.735879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" event={"ID":"5bd246b1-c940-4d9e-bad1-242827aa9b73","Type":"ContainerStarted","Data":"ac53149471c827b00e5609786f79d930abfe3fb6fdabdba7031a7c4cdff35940"}
Aug 05 17:36:24.736281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:24.736002    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:24.773457 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:24.773400    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" podStartSLOduration=1.773382429 podStartE2EDuration="1.773382429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:36:24.771140676 +0000 UTC m=+1443.304324922" watchObservedRunningTime="2026-08-05 17:36:24.773382429 +0000 UTC m=+1443.306566677"
Aug 05 17:36:29.191504 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:29.191474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:37.505888 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:37.505859    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:39.061119 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:39.061071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:39.074087 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:39.074056    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"]
Aug 05 17:36:39.162070 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:39.162041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:43.448079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:43.448043    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:36:53.448405 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:53.448362    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:36:54.097330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.097273    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" containerName="main" containerID="cri-o://415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360" gracePeriod=45
Aug 05 17:36:54.352590 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.352527    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:36:54.393950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.393917    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-tokenizer-cache\") pod \"ea491680-1b9d-4458-a9b9-647e8978da79\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") "
Aug 05 17:36:54.394124 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.393970    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8pfq\" (UniqueName: \"kubernetes.io/projected/ea491680-1b9d-4458-a9b9-647e8978da79-kube-api-access-l8pfq\") pod \"ea491680-1b9d-4458-a9b9-647e8978da79\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") "
Aug 05 17:36:54.394124 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.393995    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea491680-1b9d-4458-a9b9-647e8978da79-tls-certs\") pod \"ea491680-1b9d-4458-a9b9-647e8978da79\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") "
Aug 05 17:36:54.394124 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.394053    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-cachi2-cache\") pod \"ea491680-1b9d-4458-a9b9-647e8978da79\" (UID: \"ea491680-1b9d-4458-a9b9-647e8978da79\") "
Aug 05 17:36:54.394294 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.394216    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "ea491680-1b9d-4458-a9b9-647e8978da79" (UID: "ea491680-1b9d-4458-a9b9-647e8978da79"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:36:54.394454 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.394426    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "ea491680-1b9d-4458-a9b9-647e8978da79" (UID: "ea491680-1b9d-4458-a9b9-647e8978da79"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:36:54.396234 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.396213    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea491680-1b9d-4458-a9b9-647e8978da79-kube-api-access-l8pfq" (OuterVolumeSpecName: "kube-api-access-l8pfq") pod "ea491680-1b9d-4458-a9b9-647e8978da79" (UID: "ea491680-1b9d-4458-a9b9-647e8978da79"). InnerVolumeSpecName "kube-api-access-l8pfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:36:54.396329 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.396245    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea491680-1b9d-4458-a9b9-647e8978da79-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea491680-1b9d-4458-a9b9-647e8978da79" (UID: "ea491680-1b9d-4458-a9b9-647e8978da79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:36:54.495619 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.495575    2562 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-cachi2-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:36:54.495619 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.495614    2562 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea491680-1b9d-4458-a9b9-647e8978da79-tokenizer-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:36:54.495619 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.495624    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8pfq\" (UniqueName: \"kubernetes.io/projected/ea491680-1b9d-4458-a9b9-647e8978da79-kube-api-access-l8pfq\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:36:54.496053 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.495634    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea491680-1b9d-4458-a9b9-647e8978da79-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:36:54.876218 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.876187    2562 generic.go:356] "Generic (PLEG): container finished" podID="ea491680-1b9d-4458-a9b9-647e8978da79" containerID="415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360" exitCode=0
Aug 05 17:36:54.876417 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.876229    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"
Aug 05 17:36:54.876417 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.876265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" event={"ID":"ea491680-1b9d-4458-a9b9-647e8978da79","Type":"ContainerDied","Data":"415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360"}
Aug 05 17:36:54.876417 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.876334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9" event={"ID":"ea491680-1b9d-4458-a9b9-647e8978da79","Type":"ContainerDied","Data":"e82bd992b71651a4b77198c26a603ab8f20a380f876df08341de55d89bbd95ca"}
Aug 05 17:36:54.876417 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.876357    2562 scope.go:117] "RemoveContainer" containerID="415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360"
Aug 05 17:36:54.885660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.885645    2562 scope.go:117] "RemoveContainer" containerID="415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360"
Aug 05 17:36:54.885931 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:36:54.885909    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360\": container with ID starting with 415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360 not found: ID does not exist" containerID="415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360"
Aug 05 17:36:54.886034 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.885937    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360"} err="failed to get container status \"415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360\": rpc error: code = NotFound desc = could not find container \"415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360\": container with ID starting with 415e41c072df6a5be592dd3f7c3cd7e59d45012e97a04475baf909e0af47e360 not found: ID does not exist"
Aug 05 17:36:54.899646 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.899616    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"]
Aug 05 17:36:54.901749 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:54.901728    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-7957tcm9"]
Aug 05 17:36:55.742392 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:55.742364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:36:56.102153 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:56.102074    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" path="/var/lib/kubelet/pods/ea491680-1b9d-4458-a9b9-647e8978da79/volumes"
Aug 05 17:36:57.772302 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:57.772263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:36:59.257365 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:36:59.257335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:04.225608 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:04.225575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:04.980431 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:04.980398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:05.282727 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:05.282641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:10.013518 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:10.013482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:10.113978 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:10.113947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:37:21.958825 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:21.958793    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:37:21.963282 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:21.963264    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:37:21.963671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:21.963653    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:37:21.968185 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:21.968168    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:37:24.389512 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:37:24.389476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:28.650248 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:28.650213    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"]
Aug 05 17:38:28.655434 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:28.655405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:28.753648 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:28.753598    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:35.742010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:35.741974    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" podUID="5bd246b1-c940-4d9e-bad1-242827aa9b73" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:38:39.050081 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:39.050054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:39.149389 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:39.149353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:43.669525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:43.669489    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" podUID="5bd246b1-c940-4d9e-bad1-242827aa9b73" containerName="main" containerID="cri-o://b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0" gracePeriod=45
Aug 05 17:38:43.915383 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:43.915360    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:38:44.005646 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.005621    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd246b1-c940-4d9e-bad1-242827aa9b73-tls-certs\") pod \"5bd246b1-c940-4d9e-bad1-242827aa9b73\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") "
Aug 05 17:38:44.005789 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.005740    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvgs9\" (UniqueName: \"kubernetes.io/projected/5bd246b1-c940-4d9e-bad1-242827aa9b73-kube-api-access-mvgs9\") pod \"5bd246b1-c940-4d9e-bad1-242827aa9b73\" (UID: \"5bd246b1-c940-4d9e-bad1-242827aa9b73\") "
Aug 05 17:38:44.007751 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.007727    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bd246b1-c940-4d9e-bad1-242827aa9b73-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5bd246b1-c940-4d9e-bad1-242827aa9b73" (UID: "5bd246b1-c940-4d9e-bad1-242827aa9b73"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:38:44.007860 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.007830    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd246b1-c940-4d9e-bad1-242827aa9b73-kube-api-access-mvgs9" (OuterVolumeSpecName: "kube-api-access-mvgs9") pod "5bd246b1-c940-4d9e-bad1-242827aa9b73" (UID: "5bd246b1-c940-4d9e-bad1-242827aa9b73"). InnerVolumeSpecName "kube-api-access-mvgs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:38:44.106281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.106259    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvgs9\" (UniqueName: \"kubernetes.io/projected/5bd246b1-c940-4d9e-bad1-242827aa9b73-kube-api-access-mvgs9\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:38:44.106281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.106281    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd246b1-c940-4d9e-bad1-242827aa9b73-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:38:44.387885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.387802    2562 generic.go:356] "Generic (PLEG): container finished" podID="5bd246b1-c940-4d9e-bad1-242827aa9b73" containerID="b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0" exitCode=0
Aug 05 17:38:44.387885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.387855    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"
Aug 05 17:38:44.387885 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.387873    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" event={"ID":"5bd246b1-c940-4d9e-bad1-242827aa9b73","Type":"ContainerDied","Data":"b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0"}
Aug 05 17:38:44.388107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.387917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk" event={"ID":"5bd246b1-c940-4d9e-bad1-242827aa9b73","Type":"ContainerDied","Data":"ac53149471c827b00e5609786f79d930abfe3fb6fdabdba7031a7c4cdff35940"}
Aug 05 17:38:44.388107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.387943    2562 scope.go:117] "RemoveContainer" containerID="b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0"
Aug 05 17:38:44.396848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.396828    2562 scope.go:117] "RemoveContainer" containerID="b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0"
Aug 05 17:38:44.397091 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:38:44.397071    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0\": container with ID starting with b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0 not found: ID does not exist" containerID="b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0"
Aug 05 17:38:44.397135 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.397099    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0"} err="failed to get container status \"b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0\": rpc error: code = NotFound desc = could not find container \"b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0\": container with ID starting with b3f59a3b3376457e1078e145c7aa48e056beacdc75e560af7da0f229b371b2c0 not found: ID does not exist"
Aug 05 17:38:44.405145 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.405124    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"]
Aug 05 17:38:44.409206 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:44.409187    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-jprlk"]
Aug 05 17:38:46.101909 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:46.101876    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bd246b1-c940-4d9e-bad1-242827aa9b73" path="/var/lib/kubelet/pods/5bd246b1-c940-4d9e-bad1-242827aa9b73/volumes"
Aug 05 17:38:48.636065 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:48.636034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:48.734113 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:48.734077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:48.860768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:48.860734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:55.025151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:55.025123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:59.082612 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:59.082580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:38:59.179047 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:38:59.179020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.519791 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:01.519758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:01.617338 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:01.617283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:05.653703 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:05.653673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:07.158295 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:07.158263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:11.746420 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:11.746387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:11.874984 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:11.874949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:16.517180 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:16.517096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:17.271650 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:17.271600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:25.989715 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.989679    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"]
Aug 05 17:39:25.990111 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.990096    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" containerName="main"
Aug 05 17:39:25.990155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.990116    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" containerName="main"
Aug 05 17:39:25.990155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.990128    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd246b1-c940-4d9e-bad1-242827aa9b73" containerName="main"
Aug 05 17:39:25.990155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.990136    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd246b1-c940-4d9e-bad1-242827aa9b73" containerName="main"
Aug 05 17:39:25.990271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.990208    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd246b1-c940-4d9e-bad1-242827aa9b73" containerName="main"
Aug 05 17:39:25.990271 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.990217    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea491680-1b9d-4458-a9b9-647e8978da79" containerName="main"
Aug 05 17:39:25.993117 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.993101    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:25.997336 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.997294    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-4fx89\""
Aug 05 17:39:25.997336 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.997305    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Aug 05 17:39:25.997510 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.997305    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Aug 05 17:39:25.997510 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:25.997361    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Aug 05 17:39:26.007643 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.007622    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"]
Aug 05 17:39:26.028307 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.028283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:26.161404 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.161371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.161613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.161469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.161613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.161502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.161613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.161531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.161613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.161557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.161775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.161633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jjw\" (UniqueName: \"kubernetes.io/projected/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-kube-api-access-k6jjw\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.226357 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.226308    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"]
Aug 05 17:39:26.229512 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.229496    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.232691 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.232668    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-nl8xl\""
Aug 05 17:39:26.240872 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.240810    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"]
Aug 05 17:39:26.259170 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.259130    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:39:26.262337 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262413    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262570 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262456    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jjw\" (UniqueName: \"kubernetes.io/projected/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-kube-api-access-k6jjw\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262570 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262815 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.262882 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.262844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.264696 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.264676    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.265025 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.265007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.271190 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.271165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jjw\" (UniqueName: \"kubernetes.io/projected/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-kube-api-access-k6jjw\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.303085 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.303052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:26.363366 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.363066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8deae6f-3904-4b59-b8a1-6a7050b3546c-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.363366 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.363153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4f98\" (UniqueName: \"kubernetes.io/projected/b8deae6f-3904-4b59-b8a1-6a7050b3546c-kube-api-access-q4f98\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.446661 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.446637    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"]
Aug 05 17:39:26.448896 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:39:26.448868    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70ab257_ff7b_47f0_9ca7_1a42340eba4e.slice/crio-9b3ff9402c643a5547e7bd304273a7fe431d8e0973c397aa5cb8bcb1779eddf3 WatchSource:0}: Error finding container 9b3ff9402c643a5547e7bd304273a7fe431d8e0973c397aa5cb8bcb1779eddf3: Status 404 returned error can't find the container with id 9b3ff9402c643a5547e7bd304273a7fe431d8e0973c397aa5cb8bcb1779eddf3
Aug 05 17:39:26.464273 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.464245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4f98\" (UniqueName: \"kubernetes.io/projected/b8deae6f-3904-4b59-b8a1-6a7050b3546c-kube-api-access-q4f98\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.464401 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.464370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8deae6f-3904-4b59-b8a1-6a7050b3546c-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.466919 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.466894    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8deae6f-3904-4b59-b8a1-6a7050b3546c-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.479679 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.479662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4f98\" (UniqueName: \"kubernetes.io/projected/b8deae6f-3904-4b59-b8a1-6a7050b3546c-kube-api-access-q4f98\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.539017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.538976    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:26.592026 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.591987    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" event={"ID":"b70ab257-ff7b-47f0-9ca7-1a42340eba4e","Type":"ContainerStarted","Data":"576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe"}
Aug 05 17:39:26.592178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.592053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" event={"ID":"b70ab257-ff7b-47f0-9ca7-1a42340eba4e","Type":"ContainerStarted","Data":"9b3ff9402c643a5547e7bd304273a7fe431d8e0973c397aa5cb8bcb1779eddf3"}
Aug 05 17:39:26.628332 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.628270    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" podStartSLOduration=1.628254101 podStartE2EDuration="1.628254101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:39:26.626508096 +0000 UTC m=+1625.159692341" watchObservedRunningTime="2026-08-05 17:39:26.628254101 +0000 UTC m=+1625.161438345"
Aug 05 17:39:26.679918 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:26.679881    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"]
Aug 05 17:39:26.681996 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:39:26.681970    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8deae6f_3904_4b59_b8a1_6a7050b3546c.slice/crio-bacdd574b87fa08d7c3435af6f09b7ee6c3abb11c9a9bbee5c6b4f2777668a20 WatchSource:0}: Error finding container bacdd574b87fa08d7c3435af6f09b7ee6c3abb11c9a9bbee5c6b4f2777668a20: Status 404 returned error can't find the container with id bacdd574b87fa08d7c3435af6f09b7ee6c3abb11c9a9bbee5c6b4f2777668a20
Aug 05 17:39:27.303778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:27.303693    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:27.598041 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:27.597939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" event={"ID":"b8deae6f-3904-4b59-b8a1-6a7050b3546c","Type":"ContainerStarted","Data":"347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26"}
Aug 05 17:39:27.598041 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:27.598023    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:39:27.598295 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:27.598066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" event={"ID":"b8deae6f-3904-4b59-b8a1-6a7050b3546c","Type":"ContainerStarted","Data":"bacdd574b87fa08d7c3435af6f09b7ee6c3abb11c9a9bbee5c6b4f2777668a20"}
Aug 05 17:39:27.621457 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:27.621406    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" podStartSLOduration=1.621389347 podStartE2EDuration="1.621389347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:39:27.618235868 +0000 UTC m=+1626.151420113" watchObservedRunningTime="2026-08-05 17:39:27.621389347 +0000 UTC m=+1626.154573613"
Aug 05 17:39:36.304106 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:36.304068    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:36.316352 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:36.316329    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:36.659577 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:36.659501    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:39:58.603712 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:39:58.603681    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:40:23.053198 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:23.053166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:26.426176 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.426142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:26.447696 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.447664    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"]
Aug 05 17:40:26.453452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.453425    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"]
Aug 05 17:40:26.461925 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:40:26.461889    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:40:26.461925 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:40:26.461925 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:40:26.461925 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe" cmd=["/bin/sleep","15"]
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.461941    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" message="" err=<
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:40:26.461954    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerName="main" containerID="cri-o://576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe"
Aug 05 17:40:26.462182 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.462002    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerName="main" containerID="cri-o://576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe" gracePeriod=60
Aug 05 17:40:26.498292 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.498257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:26.649160 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.649119    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": dial tcp 10.132.0.45:8000: connect: connection refused"
Aug 05 17:40:26.888998 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.888969    2562 generic.go:356] "Generic (PLEG): container finished" podID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerID="576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe" exitCode=0
Aug 05 17:40:26.889173 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:26.889032    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" event={"ID":"b70ab257-ff7b-47f0-9ca7-1a42340eba4e","Type":"ContainerDied","Data":"576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe"}
Aug 05 17:40:27.201236 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.201205    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:40:27.369350 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369298    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-home\") pod \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") "
Aug 05 17:40:27.369540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369401    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6jjw\" (UniqueName: \"kubernetes.io/projected/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-kube-api-access-k6jjw\") pod \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") "
Aug 05 17:40:27.369540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369457    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tls-certs\") pod \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") "
Aug 05 17:40:27.369540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369480    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tmp-dir\") pod \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") "
Aug 05 17:40:27.369540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369499    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-model-cache\") pod \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") "
Aug 05 17:40:27.369540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369516    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-dshm\") pod \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\" (UID: \"b70ab257-ff7b-47f0-9ca7-1a42340eba4e\") "
Aug 05 17:40:27.369801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369577    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-home" (OuterVolumeSpecName: "home") pod "b70ab257-ff7b-47f0-9ca7-1a42340eba4e" (UID: "b70ab257-ff7b-47f0-9ca7-1a42340eba4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:40:27.369801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369726    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:27.369801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369726    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b70ab257-ff7b-47f0-9ca7-1a42340eba4e" (UID: "b70ab257-ff7b-47f0-9ca7-1a42340eba4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:40:27.369801 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.369764    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "b70ab257-ff7b-47f0-9ca7-1a42340eba4e" (UID: "b70ab257-ff7b-47f0-9ca7-1a42340eba4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:40:27.371776 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.371752    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b70ab257-ff7b-47f0-9ca7-1a42340eba4e" (UID: "b70ab257-ff7b-47f0-9ca7-1a42340eba4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:40:27.371776 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.371761    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-dshm" (OuterVolumeSpecName: "dshm") pod "b70ab257-ff7b-47f0-9ca7-1a42340eba4e" (UID: "b70ab257-ff7b-47f0-9ca7-1a42340eba4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:40:27.371917 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.371757    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-kube-api-access-k6jjw" (OuterVolumeSpecName: "kube-api-access-k6jjw") pod "b70ab257-ff7b-47f0-9ca7-1a42340eba4e" (UID: "b70ab257-ff7b-47f0-9ca7-1a42340eba4e"). InnerVolumeSpecName "kube-api-access-k6jjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:40:27.470682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.470652    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:27.470682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.470677    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:27.470682 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.470686    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:27.471099 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.470694    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:27.471099 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.470703    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6jjw\" (UniqueName: \"kubernetes.io/projected/b70ab257-ff7b-47f0-9ca7-1a42340eba4e-kube-api-access-k6jjw\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:27.895117 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.895041    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"
Aug 05 17:40:27.895277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.895043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv" event={"ID":"b70ab257-ff7b-47f0-9ca7-1a42340eba4e","Type":"ContainerDied","Data":"9b3ff9402c643a5547e7bd304273a7fe431d8e0973c397aa5cb8bcb1779eddf3"}
Aug 05 17:40:27.895277 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.895178    2562 scope.go:117] "RemoveContainer" containerID="576e84e0c8639a2694a5b5254bba62b19cc936528d2d696017cc307f46084afe"
Aug 05 17:40:27.918251 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.918228    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"]
Aug 05 17:40:27.922975 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:27.922952    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-prefill-b58fts7nv"]
Aug 05 17:40:28.102829 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:28.102798    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" path="/var/lib/kubelet/pods/b70ab257-ff7b-47f0-9ca7-1a42340eba4e/volumes"
Aug 05 17:40:28.603640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:28.603592    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:40:38.604037 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:38.603987    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:40:41.467537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.467498    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerName="main" containerID="cri-o://347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26" gracePeriod=45
Aug 05 17:40:41.712507 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.712482    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:40:41.783035 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.782951    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4f98\" (UniqueName: \"kubernetes.io/projected/b8deae6f-3904-4b59-b8a1-6a7050b3546c-kube-api-access-q4f98\") pod \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") "
Aug 05 17:40:41.783035 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.782989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8deae6f-3904-4b59-b8a1-6a7050b3546c-tls-certs\") pod \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\" (UID: \"b8deae6f-3904-4b59-b8a1-6a7050b3546c\") "
Aug 05 17:40:41.784992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.784960    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8deae6f-3904-4b59-b8a1-6a7050b3546c-kube-api-access-q4f98" (OuterVolumeSpecName: "kube-api-access-q4f98") pod "b8deae6f-3904-4b59-b8a1-6a7050b3546c" (UID: "b8deae6f-3904-4b59-b8a1-6a7050b3546c"). InnerVolumeSpecName "kube-api-access-q4f98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:40:41.785094 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.785012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8deae6f-3904-4b59-b8a1-6a7050b3546c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8deae6f-3904-4b59-b8a1-6a7050b3546c" (UID: "b8deae6f-3904-4b59-b8a1-6a7050b3546c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:40:41.883583 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.883536    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4f98\" (UniqueName: \"kubernetes.io/projected/b8deae6f-3904-4b59-b8a1-6a7050b3546c-kube-api-access-q4f98\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:41.883583 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.883579    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8deae6f-3904-4b59-b8a1-6a7050b3546c-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:40:41.961795 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.961759    2562 generic.go:356] "Generic (PLEG): container finished" podID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerID="347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26" exitCode=0
Aug 05 17:40:41.961956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.961800    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"
Aug 05 17:40:41.961956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.961804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" event={"ID":"b8deae6f-3904-4b59-b8a1-6a7050b3546c","Type":"ContainerDied","Data":"347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26"}
Aug 05 17:40:41.961956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.961926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f" event={"ID":"b8deae6f-3904-4b59-b8a1-6a7050b3546c","Type":"ContainerDied","Data":"bacdd574b87fa08d7c3435af6f09b7ee6c3abb11c9a9bbee5c6b4f2777668a20"}
Aug 05 17:40:41.961956 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.961950    2562 scope.go:117] "RemoveContainer" containerID="347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26"
Aug 05 17:40:41.970809 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.970794    2562 scope.go:117] "RemoveContainer" containerID="347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26"
Aug 05 17:40:41.971057 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:40:41.971030    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26\": container with ID starting with 347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26 not found: ID does not exist" containerID="347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26"
Aug 05 17:40:41.971160 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.971060    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26"} err="failed to get container status \"347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26\": rpc error: code = NotFound desc = could not find container \"347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26\": container with ID starting with 347e825d2c271cda7b1a13b220e9bd42531ea1240c151072f233475e629bff26 not found: ID does not exist"
Aug 05 17:40:41.984921 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.984899    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"]
Aug 05 17:40:41.989648 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:41.989624    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedkgn2f"]
Aug 05 17:40:42.102646 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:42.102568    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" path="/var/lib/kubelet/pods/b8deae6f-3904-4b59-b8a1-6a7050b3546c/volumes"
Aug 05 17:40:44.449946 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:44.449868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:44.544858 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:44.544822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:47.563887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.563852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:47.885441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.885348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:47.899859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.899830    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"]
Aug 05 17:40:47.900170 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.900159    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerName="main"
Aug 05 17:40:47.900209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.900171    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerName="main"
Aug 05 17:40:47.900209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.900179    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerName="main"
Aug 05 17:40:47.900209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.900186    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerName="main"
Aug 05 17:40:47.900302 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.900258    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70ab257-ff7b-47f0-9ca7-1a42340eba4e" containerName="main"
Aug 05 17:40:47.900302 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.900268    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8deae6f-3904-4b59-b8a1-6a7050b3546c" containerName="main"
Aug 05 17:40:47.904590 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.904571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:47.908713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.908695    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Aug 05 17:40:47.910054 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.910036    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Aug 05 17:40:47.910164 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.910066    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-xnswn\""
Aug 05 17:40:47.910164 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.910076    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Aug 05 17:40:47.910164 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.910112    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-qbppr\""
Aug 05 17:40:47.935040 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.935015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:47.935148 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.935064    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljwrm\" (UniqueName: \"kubernetes.io/projected/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-kube-api-access-ljwrm\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:47.958553 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:47.958526    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"]
Aug 05 17:40:48.024535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.024507    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:48.035692 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.035668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljwrm\" (UniqueName: \"kubernetes.io/projected/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-kube-api-access-ljwrm\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:48.035773 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.035736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:48.038194 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.038174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:48.055792 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.055767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljwrm\" (UniqueName: \"kubernetes.io/projected/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-kube-api-access-ljwrm\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:48.214514 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.214485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:48.357018 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:48.356994    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"]
Aug 05 17:40:48.358901 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:40:48.358873    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900abd18_200d_49b2_bcc8_e45ec2e3c8b9.slice/crio-1e8e60b8a8a67c03bdf620adf6c6f05dd1304b37ee75d25c660f375b50446252 WatchSource:0}: Error finding container 1e8e60b8a8a67c03bdf620adf6c6f05dd1304b37ee75d25c660f375b50446252: Status 404 returned error can't find the container with id 1e8e60b8a8a67c03bdf620adf6c6f05dd1304b37ee75d25c660f375b50446252
Aug 05 17:40:49.000639 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:49.000607    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" event={"ID":"900abd18-200d-49b2-bcc8-e45ec2e3c8b9","Type":"ContainerStarted","Data":"be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465"}
Aug 05 17:40:49.001036 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:49.000660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" event={"ID":"900abd18-200d-49b2-bcc8-e45ec2e3c8b9","Type":"ContainerStarted","Data":"1e8e60b8a8a67c03bdf620adf6c6f05dd1304b37ee75d25c660f375b50446252"}
Aug 05 17:40:49.001036 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:49.000726    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:40:52.667378 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:52.667346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:58.894761 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:58.894724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:40:58.996119 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:40:58.996084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:15.549807 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:15.549770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:20.010857 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:20.010821    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:41:20.029740 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:20.029690    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" podStartSLOduration=33.029674842 podStartE2EDuration="33.029674842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:40:49.028516993 +0000 UTC m=+1707.561701239" watchObservedRunningTime="2026-08-05 17:41:20.029674842 +0000 UTC m=+1738.562859087"
Aug 05 17:41:35.277179 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:35.277145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:35.341859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:35.341826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:43.011088 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:43.011050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:47.933858 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:47.933823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:48.037301 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:48.037269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:48.128433 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:48.128404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:48.197878 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:48.197852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:49.119120 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:49.119086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:41:53.007999 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:41:53.007966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:02.315659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.315613    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"]
Aug 05 17:42:02.319208 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.319188    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.322504 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.322485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"default-dockercfg-ksxsl\""
Aug 05 17:42:02.323411 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.323388    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"test-div-v1-kserve-self-signed-certs\""
Aug 05 17:42:02.323517 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.323433    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"openshift-service-ca.crt\""
Aug 05 17:42:02.323517 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.323491    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"kube-root-ca.crt\""
Aug 05 17:42:02.333010 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.332988    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"]
Aug 05 17:42:02.344744 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.344719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:02.454471 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.454442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-dshm\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.454641 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.454478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jccwq\" (UniqueName: \"kubernetes.io/projected/afe06122-7e7f-4b02-aee6-93f538f50793-kube-api-access-jccwq\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.454641 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.454501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-model-cache\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.454641 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.454611    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-home\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.454641 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.454637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-tmp-dir\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.454789 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.454663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afe06122-7e7f-4b02-aee6-93f538f50793-tls-certs\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.555999 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.555959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-dshm\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.555999 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jccwq\" (UniqueName: \"kubernetes.io/projected/afe06122-7e7f-4b02-aee6-93f538f50793-kube-api-access-jccwq\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-model-cache\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-home\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-tmp-dir\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afe06122-7e7f-4b02-aee6-93f538f50793-tls-certs\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556563 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556531    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-home\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556694 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-model-cache\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.556694 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.556593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-tmp-dir\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.558396 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.558377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-dshm\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.558694 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.558674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afe06122-7e7f-4b02-aee6-93f538f50793-tls-certs\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.568174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.568118    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jccwq\" (UniqueName: \"kubernetes.io/projected/afe06122-7e7f-4b02-aee6-93f538f50793-kube-api-access-jccwq\") pod \"test-div-v1-kserve-fb566d667-45qc8\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.629820 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.629772    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:02.779024 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.779001    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"]
Aug 05 17:42:02.782179 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:42:02.782152    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe06122_7e7f_4b02_aee6_93f538f50793.slice/crio-40e840deed93aab57760b8f1d859309a8e2ad524c5a49e4cfc37687506380260 WatchSource:0}: Error finding container 40e840deed93aab57760b8f1d859309a8e2ad524c5a49e4cfc37687506380260: Status 404 returned error can't find the container with id 40e840deed93aab57760b8f1d859309a8e2ad524c5a49e4cfc37687506380260
Aug 05 17:42:02.784209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:02.784193    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:42:03.356528 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:03.356487    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8" event={"ID":"afe06122-7e7f-4b02-aee6-93f538f50793","Type":"ContainerStarted","Data":"40e840deed93aab57760b8f1d859309a8e2ad524c5a49e4cfc37687506380260"}
Aug 05 17:42:04.363159 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:04.363125    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8" event={"ID":"afe06122-7e7f-4b02-aee6-93f538f50793","Type":"ContainerStarted","Data":"f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5"}
Aug 05 17:42:04.386207 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:04.386153    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8" podStartSLOduration=1.341937766 podStartE2EDuration="2.386138435s" totalImagesPullingTime="1.044200669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:42:04.383591606 +0000 UTC m=+1782.916775853" watchObservedRunningTime="2026-08-05 17:42:04.386138435 +0000 UTC m=+1782.919322679"
Aug 05 17:42:04.630533 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:04.630445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:09.921033 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:09.921000    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"]
Aug 05 17:42:09.924555 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:09.924536    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:09.927189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:09.927166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-name-divergence-75f1ef44\"/\"test-div-v2-kserve-self-signed-certs\""
Aug 05 17:42:09.937991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:09.937967    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"]
Aug 05 17:42:09.951818 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:09.951793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:10.028728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.028686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-dshm\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.028728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.028728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-home\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.028939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.028745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/280ea35b-e712-46c6-9829-2ebd17e76500-tls-certs\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.028939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.028841    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-tmp-dir\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.028939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.028869    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kchv\" (UniqueName: \"kubernetes.io/projected/280ea35b-e712-46c6-9829-2ebd17e76500-kube-api-access-9kchv\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.028939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.028931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-model-cache\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130153 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-tmp-dir\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130153 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kchv\" (UniqueName: \"kubernetes.io/projected/280ea35b-e712-46c6-9829-2ebd17e76500-kube-api-access-9kchv\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130463 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-model-cache\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130463 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-dshm\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130463 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-home\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130463 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/280ea35b-e712-46c6-9829-2ebd17e76500-tls-certs\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130629 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-tmp-dir\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130680 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130653    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-home\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.130723 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.130661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-model-cache\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.132584 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.132556    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-dshm\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.132741 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.132724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/280ea35b-e712-46c6-9829-2ebd17e76500-tls-certs\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.141478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.141454    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kchv\" (UniqueName: \"kubernetes.io/projected/280ea35b-e712-46c6-9829-2ebd17e76500-kube-api-access-9kchv\") pod \"test-div-v2-kserve-5f6fdb6b77-8r4vb\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") " pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.234950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.234916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:10.375966 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.375935    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"]
Aug 05 17:42:10.377308 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:42:10.377280    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280ea35b_e712_46c6_9829_2ebd17e76500.slice/crio-943393a6b009f40b0edb971c7599ca4157a2614465944ca5141358f28e1c2479 WatchSource:0}: Error finding container 943393a6b009f40b0edb971c7599ca4157a2614465944ca5141358f28e1c2479: Status 404 returned error can't find the container with id 943393a6b009f40b0edb971c7599ca4157a2614465944ca5141358f28e1c2479
Aug 05 17:42:10.393996 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:10.393968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" event={"ID":"280ea35b-e712-46c6-9829-2ebd17e76500","Type":"ContainerStarted","Data":"943393a6b009f40b0edb971c7599ca4157a2614465944ca5141358f28e1c2479"}
Aug 05 17:42:11.400537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:11.400504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" event={"ID":"280ea35b-e712-46c6-9829-2ebd17e76500","Type":"ContainerStarted","Data":"97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35"}
Aug 05 17:42:11.424225 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:11.424167    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" podStartSLOduration=2.424151857 podStartE2EDuration="2.424151857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:42:11.422098933 +0000 UTC m=+1789.955283179" watchObservedRunningTime="2026-08-05 17:42:11.424151857 +0000 UTC m=+1789.957336102"
Aug 05 17:42:12.235196 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:12.235160    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:12.630736 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:12.630639    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:12.643190 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:12.643166    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:13.421883 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:13.421850    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:20.235378 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:20.235350    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:20.248184 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:20.248154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:20.454887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:20.454854    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:21.990178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:21.990147    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:42:21.993739 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:21.993718    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:42:21.994544 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:21.994528    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:42:21.998734 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:21.998716    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:42:30.448243 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:30.448215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:31.632878 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:31.632834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:31.971450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:31.971420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:32.068142 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:32.068112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:36.269066 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:36.269032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:36.509355 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:36.509293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:36.655416 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:36.655304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:36.690104 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:36.690071    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"]
Aug 05 17:42:36.823902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:36.823871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:39.587477 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:39.587440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:39.669251 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:39.669216    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"]
Aug 05 17:42:39.745561 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:39.745528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:39.838601 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:39.838523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:40.010775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:40.010737    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:42:40.350139 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:40.350104    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"]
Aug 05 17:42:40.352509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:40.352482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:44.651955 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:44.651905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:50.010613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:50.010577    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:42:51.709635 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:51.709583    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerName="main" containerID="cri-o://be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465" gracePeriod=45
Aug 05 17:42:51.839388 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:51.839359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:51.961209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:51.961148    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:42:52.110595 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.110566    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljwrm\" (UniqueName: \"kubernetes.io/projected/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-kube-api-access-ljwrm\") pod \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") "
Aug 05 17:42:52.110773 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.110607    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-tls-certs\") pod \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\" (UID: \"900abd18-200d-49b2-bcc8-e45ec2e3c8b9\") "
Aug 05 17:42:52.115079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.115045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "900abd18-200d-49b2-bcc8-e45ec2e3c8b9" (UID: "900abd18-200d-49b2-bcc8-e45ec2e3c8b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:42:52.115189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.115119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-kube-api-access-ljwrm" (OuterVolumeSpecName: "kube-api-access-ljwrm") pod "900abd18-200d-49b2-bcc8-e45ec2e3c8b9" (UID: "900abd18-200d-49b2-bcc8-e45ec2e3c8b9"). InnerVolumeSpecName "kube-api-access-ljwrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:42:52.212100 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.212030    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljwrm\" (UniqueName: \"kubernetes.io/projected/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-kube-api-access-ljwrm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:52.212100 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.212054    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900abd18-200d-49b2-bcc8-e45ec2e3c8b9-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:52.602220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.602139    2562 generic.go:356] "Generic (PLEG): container finished" podID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerID="be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465" exitCode=0
Aug 05 17:42:52.602220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.602182    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"
Aug 05 17:42:52.602220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.602210    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" event={"ID":"900abd18-200d-49b2-bcc8-e45ec2e3c8b9","Type":"ContainerDied","Data":"be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465"}
Aug 05 17:42:52.602509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.602267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg" event={"ID":"900abd18-200d-49b2-bcc8-e45ec2e3c8b9","Type":"ContainerDied","Data":"1e8e60b8a8a67c03bdf620adf6c6f05dd1304b37ee75d25c660f375b50446252"}
Aug 05 17:42:52.602509 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.602297    2562 scope.go:117] "RemoveContainer" containerID="be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465"
Aug 05 17:42:52.611353 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.611330    2562 scope.go:117] "RemoveContainer" containerID="be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465"
Aug 05 17:42:52.611633 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:42:52.611599    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465\": container with ID starting with be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465 not found: ID does not exist" containerID="be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465"
Aug 05 17:42:52.611694 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.611641    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465"} err="failed to get container status \"be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465\": rpc error: code = NotFound desc = could not find container \"be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465\": container with ID starting with be290ae20a931a444f1240aeadf4a78a3718d1da35838f6149df3a298ba4e465 not found: ID does not exist"
Aug 05 17:42:52.629861 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.629832    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"]
Aug 05 17:42:52.634690 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:52.634669    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65hvmrg"]
Aug 05 17:42:54.102113 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:54.102077    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" path="/var/lib/kubelet/pods/900abd18-200d-49b2-bcc8-e45ec2e3c8b9/volumes"
Aug 05 17:42:54.557567 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:54.557528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:54.659203 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:54.659173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:42:54.688621 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:54.688585    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8" podUID="afe06122-7e7f-4b02-aee6-93f538f50793" containerName="main" containerID="cri-o://f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5" gracePeriod=45
Aug 05 17:42:54.937387 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:54.937360    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:55.038756 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.038724    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-home\") pod \"afe06122-7e7f-4b02-aee6-93f538f50793\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") "
Aug 05 17:42:55.038756 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.038758    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-tmp-dir\") pod \"afe06122-7e7f-4b02-aee6-93f538f50793\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") "
Aug 05 17:42:55.039002 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.038786    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jccwq\" (UniqueName: \"kubernetes.io/projected/afe06122-7e7f-4b02-aee6-93f538f50793-kube-api-access-jccwq\") pod \"afe06122-7e7f-4b02-aee6-93f538f50793\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") "
Aug 05 17:42:55.039002 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.038808    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afe06122-7e7f-4b02-aee6-93f538f50793-tls-certs\") pod \"afe06122-7e7f-4b02-aee6-93f538f50793\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") "
Aug 05 17:42:55.039002 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.038954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-dshm\") pod \"afe06122-7e7f-4b02-aee6-93f538f50793\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") "
Aug 05 17:42:55.039002 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.038988    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-model-cache\") pod \"afe06122-7e7f-4b02-aee6-93f538f50793\" (UID: \"afe06122-7e7f-4b02-aee6-93f538f50793\") "
Aug 05 17:42:55.039209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.039057    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-home" (OuterVolumeSpecName: "home") pod "afe06122-7e7f-4b02-aee6-93f538f50793" (UID: "afe06122-7e7f-4b02-aee6-93f538f50793"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.039209 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.039064    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "afe06122-7e7f-4b02-aee6-93f538f50793" (UID: "afe06122-7e7f-4b02-aee6-93f538f50793"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.039304 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.039290    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.039388 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.039306    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.039388 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.039306    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-model-cache" (OuterVolumeSpecName: "model-cache") pod "afe06122-7e7f-4b02-aee6-93f538f50793" (UID: "afe06122-7e7f-4b02-aee6-93f538f50793"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.041131 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.041106    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe06122-7e7f-4b02-aee6-93f538f50793-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "afe06122-7e7f-4b02-aee6-93f538f50793" (UID: "afe06122-7e7f-4b02-aee6-93f538f50793"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:42:55.041226 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.041145    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe06122-7e7f-4b02-aee6-93f538f50793-kube-api-access-jccwq" (OuterVolumeSpecName: "kube-api-access-jccwq") pod "afe06122-7e7f-4b02-aee6-93f538f50793" (UID: "afe06122-7e7f-4b02-aee6-93f538f50793"). InnerVolumeSpecName "kube-api-access-jccwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:42:55.041226 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.041187    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-dshm" (OuterVolumeSpecName: "dshm") pod "afe06122-7e7f-4b02-aee6-93f538f50793" (UID: "afe06122-7e7f-4b02-aee6-93f538f50793"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.140329 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.140232    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.140329 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.140257    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afe06122-7e7f-4b02-aee6-93f538f50793-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.140329 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.140267    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jccwq\" (UniqueName: \"kubernetes.io/projected/afe06122-7e7f-4b02-aee6-93f538f50793-kube-api-access-jccwq\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.140329 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.140276    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afe06122-7e7f-4b02-aee6-93f538f50793-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.369260 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.369217    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" podUID="280ea35b-e712-46c6-9829-2ebd17e76500" containerName="main" containerID="cri-o://97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35" gracePeriod=45
Aug 05 17:42:55.444110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.444066    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" podUID="280ea35b-e712-46c6-9829-2ebd17e76500" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": dial tcp 10.132.0.49:8000: connect: connection refused"
Aug 05 17:42:55.606659 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.606620    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:55.625326 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.625281    2562 generic.go:356] "Generic (PLEG): container finished" podID="afe06122-7e7f-4b02-aee6-93f538f50793" containerID="f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5" exitCode=0
Aug 05 17:42:55.625492 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.625351    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"
Aug 05 17:42:55.625492 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.625351    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8" event={"ID":"afe06122-7e7f-4b02-aee6-93f538f50793","Type":"ContainerDied","Data":"f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5"}
Aug 05 17:42:55.625492 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.625438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8" event={"ID":"afe06122-7e7f-4b02-aee6-93f538f50793","Type":"ContainerDied","Data":"40e840deed93aab57760b8f1d859309a8e2ad524c5a49e4cfc37687506380260"}
Aug 05 17:42:55.625492 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.625468    2562 scope.go:117] "RemoveContainer" containerID="f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5"
Aug 05 17:42:55.626729 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.626705    2562 generic.go:356] "Generic (PLEG): container finished" podID="280ea35b-e712-46c6-9829-2ebd17e76500" containerID="97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35" exitCode=0
Aug 05 17:42:55.626912 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.626779    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"
Aug 05 17:42:55.626912 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.626803    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" event={"ID":"280ea35b-e712-46c6-9829-2ebd17e76500","Type":"ContainerDied","Data":"97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35"}
Aug 05 17:42:55.626912 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.626866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb" event={"ID":"280ea35b-e712-46c6-9829-2ebd17e76500","Type":"ContainerDied","Data":"943393a6b009f40b0edb971c7599ca4157a2614465944ca5141358f28e1c2479"}
Aug 05 17:42:55.636427 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.636410    2562 scope.go:117] "RemoveContainer" containerID="f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5"
Aug 05 17:42:55.636698 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:42:55.636678    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5\": container with ID starting with f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5 not found: ID does not exist" containerID="f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5"
Aug 05 17:42:55.636762 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.636704    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5"} err="failed to get container status \"f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5\": rpc error: code = NotFound desc = could not find container \"f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5\": container with ID starting with f6f76fec58194ba23e80e186b0d8d929c915b9ab0e5e87642e7cb3ab895ba2a5 not found: ID does not exist"
Aug 05 17:42:55.636762 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.636719    2562 scope.go:117] "RemoveContainer" containerID="97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35"
Aug 05 17:42:55.644644 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.644615    2562 scope.go:117] "RemoveContainer" containerID="97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35"
Aug 05 17:42:55.644908 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:42:55.644887    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35\": container with ID starting with 97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35 not found: ID does not exist" containerID="97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35"
Aug 05 17:42:55.644987 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.644915    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35"} err="failed to get container status \"97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35\": rpc error: code = NotFound desc = could not find container \"97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35\": container with ID starting with 97475644c17fc743c7820403099a21ad91ce88c9f0309af536cc7aedd40dee35 not found: ID does not exist"
Aug 05 17:42:55.653120 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.653094    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"]
Aug 05 17:42:55.657677 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.657657    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v1-kserve-fb566d667-45qc8"]
Aug 05 17:42:55.745189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745164    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-model-cache\") pod \"280ea35b-e712-46c6-9829-2ebd17e76500\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") "
Aug 05 17:42:55.745347 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745207    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-home\") pod \"280ea35b-e712-46c6-9829-2ebd17e76500\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") "
Aug 05 17:42:55.745347 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745261    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-tmp-dir\") pod \"280ea35b-e712-46c6-9829-2ebd17e76500\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") "
Aug 05 17:42:55.745465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745405    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kchv\" (UniqueName: \"kubernetes.io/projected/280ea35b-e712-46c6-9829-2ebd17e76500-kube-api-access-9kchv\") pod \"280ea35b-e712-46c6-9829-2ebd17e76500\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") "
Aug 05 17:42:55.745465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745437    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/280ea35b-e712-46c6-9829-2ebd17e76500-tls-certs\") pod \"280ea35b-e712-46c6-9829-2ebd17e76500\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") "
Aug 05 17:42:55.745465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745452    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-model-cache" (OuterVolumeSpecName: "model-cache") pod "280ea35b-e712-46c6-9829-2ebd17e76500" (UID: "280ea35b-e712-46c6-9829-2ebd17e76500"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.745605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745486    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-dshm\") pod \"280ea35b-e712-46c6-9829-2ebd17e76500\" (UID: \"280ea35b-e712-46c6-9829-2ebd17e76500\") "
Aug 05 17:42:55.745605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745548    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-home" (OuterVolumeSpecName: "home") pod "280ea35b-e712-46c6-9829-2ebd17e76500" (UID: "280ea35b-e712-46c6-9829-2ebd17e76500"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.745605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745557    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "280ea35b-e712-46c6-9829-2ebd17e76500" (UID: "280ea35b-e712-46c6-9829-2ebd17e76500"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.745728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745703    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.745728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745716    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.745728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.745726    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.747478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.747455    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-dshm" (OuterVolumeSpecName: "dshm") pod "280ea35b-e712-46c6-9829-2ebd17e76500" (UID: "280ea35b-e712-46c6-9829-2ebd17e76500"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:55.747540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.747526    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/280ea35b-e712-46c6-9829-2ebd17e76500-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "280ea35b-e712-46c6-9829-2ebd17e76500" (UID: "280ea35b-e712-46c6-9829-2ebd17e76500"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:42:55.747604 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.747590    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280ea35b-e712-46c6-9829-2ebd17e76500-kube-api-access-9kchv" (OuterVolumeSpecName: "kube-api-access-9kchv") pod "280ea35b-e712-46c6-9829-2ebd17e76500" (UID: "280ea35b-e712-46c6-9829-2ebd17e76500"). InnerVolumeSpecName "kube-api-access-9kchv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:42:55.846503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.846457    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kchv\" (UniqueName: \"kubernetes.io/projected/280ea35b-e712-46c6-9829-2ebd17e76500-kube-api-access-9kchv\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.846503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.846497    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/280ea35b-e712-46c6-9829-2ebd17e76500-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.846503 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.846508    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/280ea35b-e712-46c6-9829-2ebd17e76500-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:42:55.950575 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.950545    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"]
Aug 05 17:42:55.955141 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:55.955084    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-name-divergence-75f1ef44/test-div-v2-kserve-5f6fdb6b77-8r4vb"]
Aug 05 17:42:56.102306 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:56.102269    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="280ea35b-e712-46c6-9829-2ebd17e76500" path="/var/lib/kubelet/pods/280ea35b-e712-46c6-9829-2ebd17e76500/volumes"
Aug 05 17:42:56.102778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:56.102757    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afe06122-7e7f-4b02-aee6-93f538f50793" path="/var/lib/kubelet/pods/afe06122-7e7f-4b02-aee6-93f538f50793/volumes"
Aug 05 17:42:57.668687 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:42:57.668653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:01.618516 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:01.618485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:07.935137 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:07.935102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:07.998066 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:07.998036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:20.659356 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.659301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:20.914821 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.914747    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"]
Aug 05 17:43:20.915133 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915118    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerName="main"
Aug 05 17:43:20.915189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915136    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerName="main"
Aug 05 17:43:20.915189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915146    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe06122-7e7f-4b02-aee6-93f538f50793" containerName="main"
Aug 05 17:43:20.915189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915153    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe06122-7e7f-4b02-aee6-93f538f50793" containerName="main"
Aug 05 17:43:20.915189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915174    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280ea35b-e712-46c6-9829-2ebd17e76500" containerName="main"
Aug 05 17:43:20.915189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915180    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="280ea35b-e712-46c6-9829-2ebd17e76500" containerName="main"
Aug 05 17:43:20.915377 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915234    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="900abd18-200d-49b2-bcc8-e45ec2e3c8b9" containerName="main"
Aug 05 17:43:20.915377 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915242    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="280ea35b-e712-46c6-9829-2ebd17e76500" containerName="main"
Aug 05 17:43:20.915377 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.915250    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe06122-7e7f-4b02-aee6-93f538f50793" containerName="main"
Aug 05 17:43:20.918348 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.918327    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:20.922436 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.922409    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Aug 05 17:43:20.922575 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.922484    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Aug 05 17:43:20.922575 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.922492    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-jwjml\""
Aug 05 17:43:20.922575 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.922492    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Aug 05 17:43:20.922759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.922589    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-2mffk\""
Aug 05 17:43:20.927827 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.927779    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:20.928934 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.928912    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"]
Aug 05 17:43:20.945454 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.945430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4f5\" (UniqueName: \"kubernetes.io/projected/92efd759-1ef4-4f67-b0e8-b13448a2c084-kube-api-access-5t4f5\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:20.945552 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:20.945471    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92efd759-1ef4-4f67-b0e8-b13448a2c084-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.025414 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.025384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:21.046704 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.046680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92efd759-1ef4-4f67-b0e8-b13448a2c084-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.046788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.046747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4f5\" (UniqueName: \"kubernetes.io/projected/92efd759-1ef4-4f67-b0e8-b13448a2c084-kube-api-access-5t4f5\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.049118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.049098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92efd759-1ef4-4f67-b0e8-b13448a2c084-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.061272 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.061253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4f5\" (UniqueName: \"kubernetes.io/projected/92efd759-1ef4-4f67-b0e8-b13448a2c084-kube-api-access-5t4f5\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.229369 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.229337    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.360161 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.360123    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"]
Aug 05 17:43:21.363538 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:43:21.363512    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92efd759_1ef4_4f67_b0e8_b13448a2c084.slice/crio-a400cfb306bbd1b8fb5bb6cbff243ffd52a2be168bdd3fe82ca2d7c14e5f7bc0 WatchSource:0}: Error finding container a400cfb306bbd1b8fb5bb6cbff243ffd52a2be168bdd3fe82ca2d7c14e5f7bc0: Status 404 returned error can't find the container with id a400cfb306bbd1b8fb5bb6cbff243ffd52a2be168bdd3fe82ca2d7c14e5f7bc0
Aug 05 17:43:21.751192 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.751159    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" event={"ID":"92efd759-1ef4-4f67-b0e8-b13448a2c084","Type":"ContainerStarted","Data":"27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74"}
Aug 05 17:43:21.751655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.751213    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" event={"ID":"92efd759-1ef4-4f67-b0e8-b13448a2c084","Type":"ContainerStarted","Data":"a400cfb306bbd1b8fb5bb6cbff243ffd52a2be168bdd3fe82ca2d7c14e5f7bc0"}
Aug 05 17:43:21.751655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.751263    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:21.776297 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:21.776245    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" podStartSLOduration=1.7762302509999999 podStartE2EDuration="1.776230251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:43:21.773544944 +0000 UTC m=+1860.306729189" watchObservedRunningTime="2026-08-05 17:43:21.776230251 +0000 UTC m=+1860.309414497"
Aug 05 17:43:23.205232 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:23.205190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:29.164962 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:29.164926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:29.229267 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:29.229217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:43:52.758803 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:52.758731    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:43:59.443898 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:43:59.443863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:02.936989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:02.936952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:02.957249 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:02.957211    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"]
Aug 05 17:44:03.025250 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:03.025200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.065328 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.065285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.167534 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.167503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.263580 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.263552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.684719 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.684691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.775626 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.775595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.873840 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.873803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:05.970660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:05.970630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:06.242269 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:06.242180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:10.135771 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:10.135739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:12.140220 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:12.140189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:12.758498 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:12.758462    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" podUID="92efd759-1ef4-4f67-b0e8-b13448a2c084" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:44:17.982037 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:17.981991    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" podUID="92efd759-1ef4-4f67-b0e8-b13448a2c084" containerName="main" containerID="cri-o://27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74" gracePeriod=45
Aug 05 17:44:18.218559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.218537    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:44:18.316027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.315948    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92efd759-1ef4-4f67-b0e8-b13448a2c084-tls-certs\") pod \"92efd759-1ef4-4f67-b0e8-b13448a2c084\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") "
Aug 05 17:44:18.316174 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.316035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t4f5\" (UniqueName: \"kubernetes.io/projected/92efd759-1ef4-4f67-b0e8-b13448a2c084-kube-api-access-5t4f5\") pod \"92efd759-1ef4-4f67-b0e8-b13448a2c084\" (UID: \"92efd759-1ef4-4f67-b0e8-b13448a2c084\") "
Aug 05 17:44:18.318069 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.318041    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92efd759-1ef4-4f67-b0e8-b13448a2c084-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92efd759-1ef4-4f67-b0e8-b13448a2c084" (UID: "92efd759-1ef4-4f67-b0e8-b13448a2c084"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:44:18.318196 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.318090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92efd759-1ef4-4f67-b0e8-b13448a2c084-kube-api-access-5t4f5" (OuterVolumeSpecName: "kube-api-access-5t4f5") pod "92efd759-1ef4-4f67-b0e8-b13448a2c084" (UID: "92efd759-1ef4-4f67-b0e8-b13448a2c084"). InnerVolumeSpecName "kube-api-access-5t4f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:44:18.416649 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.416618    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92efd759-1ef4-4f67-b0e8-b13448a2c084-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:44:18.416649 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:18.416645    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t4f5\" (UniqueName: \"kubernetes.io/projected/92efd759-1ef4-4f67-b0e8-b13448a2c084-kube-api-access-5t4f5\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:44:19.031245 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.031209    2562 generic.go:356] "Generic (PLEG): container finished" podID="92efd759-1ef4-4f67-b0e8-b13448a2c084" containerID="27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74" exitCode=0
Aug 05 17:44:19.031686 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.031250    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" event={"ID":"92efd759-1ef4-4f67-b0e8-b13448a2c084","Type":"ContainerDied","Data":"27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74"}
Aug 05 17:44:19.031686 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.031293    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8" event={"ID":"92efd759-1ef4-4f67-b0e8-b13448a2c084","Type":"ContainerDied","Data":"a400cfb306bbd1b8fb5bb6cbff243ffd52a2be168bdd3fe82ca2d7c14e5f7bc0"}
Aug 05 17:44:19.031686 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.031340    2562 scope.go:117] "RemoveContainer" containerID="27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74"
Aug 05 17:44:19.031686 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.031251    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"
Aug 05 17:44:19.040372 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.040350    2562 scope.go:117] "RemoveContainer" containerID="27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74"
Aug 05 17:44:19.040628 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:44:19.040609    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74\": container with ID starting with 27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74 not found: ID does not exist" containerID="27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74"
Aug 05 17:44:19.040685 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.040638    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74"} err="failed to get container status \"27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74\": rpc error: code = NotFound desc = could not find container \"27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74\": container with ID starting with 27f0b82cc9dd2d144f28b275d7f11eab35d1797c832ed1cf2d0b18f3e43c6b74 not found: ID does not exist"
Aug 05 17:44:19.056138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.056112    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"]
Aug 05 17:44:19.063653 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.063627    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-c8b9fswp8"]
Aug 05 17:44:19.194070 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.194040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:19.291999 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:19.291930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:20.102553 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:20.102523    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92efd759-1ef4-4f67-b0e8-b13448a2c084" path="/var/lib/kubelet/pods/92efd759-1ef4-4f67-b0e8-b13448a2c084/volumes"
Aug 05 17:44:22.299033 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:22.299005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:22.395419 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:22.395341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:29.090605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:29.090567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:35.176101 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:35.176060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:35.246760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:35.246731    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:48.226184 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:48.226154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:51.755783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:51.755751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:54.854404 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:54.854375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:44:54.957233 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:44:54.957206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:04.670246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:04.670204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:17.151415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.151335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:17.476113 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.476080    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"]
Aug 05 17:45:17.476775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.476751    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92efd759-1ef4-4f67-b0e8-b13448a2c084" containerName="main"
Aug 05 17:45:17.476775 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.476774    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="92efd759-1ef4-4f67-b0e8-b13448a2c084" containerName="main"
Aug 05 17:45:17.476915 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.476835    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="92efd759-1ef4-4f67-b0e8-b13448a2c084" containerName="main"
Aug 05 17:45:17.479807 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.479789    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.481435 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.481408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:17.484889 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.484844    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Aug 05 17:45:17.485011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.484903    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Aug 05 17:45:17.485011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.484927    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-epp-sa-dockercfg-4zl8r\""
Aug 05 17:45:17.485011 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.484933    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-2nw7j\""
Aug 05 17:45:17.485190 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.485125    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Aug 05 17:45:17.493971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.493946    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"]
Aug 05 17:45:17.549894 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.549863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:17.577578 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.577549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-6b477rx872\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.577711 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.577619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv2lr\" (UniqueName: \"kubernetes.io/projected/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-kube-api-access-cv2lr\") pod \"tokenizer-template-args-test-kserve-router-scheduler-6b477rx872\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.678523 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.678485    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-6b477rx872\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.678695 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.678586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv2lr\" (UniqueName: \"kubernetes.io/projected/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-kube-api-access-cv2lr\") pod \"tokenizer-template-args-test-kserve-router-scheduler-6b477rx872\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.681056 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.681033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-6b477rx872\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.692046 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.692018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv2lr\" (UniqueName: \"kubernetes.io/projected/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-kube-api-access-cv2lr\") pod \"tokenizer-template-args-test-kserve-router-scheduler-6b477rx872\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.790330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.790253    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:17.924034 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:17.924009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"]
Aug 05 17:45:17.926454 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:45:17.926423    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2019bfa_a1d4_4fbe_a96f_64535c2ac5ac.slice/crio-90554cfff2df4c6bc5a18c38328c9844d611c23ebeb6ed6e8f08d640823f890e WatchSource:0}: Error finding container 90554cfff2df4c6bc5a18c38328c9844d611c23ebeb6ed6e8f08d640823f890e: Status 404 returned error can't find the container with id 90554cfff2df4c6bc5a18c38328c9844d611c23ebeb6ed6e8f08d640823f890e
Aug 05 17:45:18.294513 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:18.294458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" event={"ID":"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac","Type":"ContainerStarted","Data":"2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9"}
Aug 05 17:45:18.294893 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:18.294532    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:18.294893 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:18.294552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" event={"ID":"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac","Type":"ContainerStarted","Data":"90554cfff2df4c6bc5a18c38328c9844d611c23ebeb6ed6e8f08d640823f890e"}
Aug 05 17:45:18.325052 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:18.325002    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" podStartSLOduration=1.324985405 podStartE2EDuration="1.324985405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:45:18.322208128 +0000 UTC m=+1976.855392374" watchObservedRunningTime="2026-08-05 17:45:18.324985405 +0000 UTC m=+1976.858169650"
Aug 05 17:45:49.300448 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:49.300419    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:45:52.289617 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:52.289587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:56.757240 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:56.757201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:56.769876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:56.769847    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"]
Aug 05 17:45:56.823574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:56.823540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:58.000983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:58.000933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:45:59.300299 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:45:59.300256    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:46:09.299789 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:09.299752    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:46:11.789371 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:11.789310    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerName="main" containerID="cri-o://2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9" gracePeriod=45
Aug 05 17:46:12.035809 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.035784    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:46:12.131554 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.131484    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv2lr\" (UniqueName: \"kubernetes.io/projected/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-kube-api-access-cv2lr\") pod \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") "
Aug 05 17:46:12.131703 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.131563    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-tls-certs\") pod \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\" (UID: \"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac\") "
Aug 05 17:46:12.133664 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.133640    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-kube-api-access-cv2lr" (OuterVolumeSpecName: "kube-api-access-cv2lr") pod "c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" (UID: "c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac"). InnerVolumeSpecName "kube-api-access-cv2lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:46:12.133751 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.133728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" (UID: "c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:46:12.232652 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.232624    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv2lr\" (UniqueName: \"kubernetes.io/projected/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-kube-api-access-cv2lr\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:46:12.232652 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.232647    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:46:12.539384 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.539353    2562 generic.go:356] "Generic (PLEG): container finished" podID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerID="2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9" exitCode=0
Aug 05 17:46:12.539526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.539409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" event={"ID":"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac","Type":"ContainerDied","Data":"2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9"}
Aug 05 17:46:12.539526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.539450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872" event={"ID":"c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac","Type":"ContainerDied","Data":"90554cfff2df4c6bc5a18c38328c9844d611c23ebeb6ed6e8f08d640823f890e"}
Aug 05 17:46:12.539526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.539470    2562 scope.go:117] "RemoveContainer" containerID="2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9"
Aug 05 17:46:12.539526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.539411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"
Aug 05 17:46:12.549003 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.548985    2562 scope.go:117] "RemoveContainer" containerID="2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9"
Aug 05 17:46:12.549243 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:46:12.549222    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9\": container with ID starting with 2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9 not found: ID does not exist" containerID="2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9"
Aug 05 17:46:12.549335 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.549250    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9"} err="failed to get container status \"2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9\": rpc error: code = NotFound desc = could not find container \"2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9\": container with ID starting with 2ede8133e4279b932ab181c4bf4c2256bcdfe3450b5fbb341641e6beda57bdc9 not found: ID does not exist"
Aug 05 17:46:12.562279 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.562257    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"]
Aug 05 17:46:12.568903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:12.568881    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-6b477rx872"]
Aug 05 17:46:14.102388 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:14.102358    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" path="/var/lib/kubelet/pods/c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac/volumes"
Aug 05 17:46:16.574246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:16.574212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:16.667430 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:16.667395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:19.385989 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:19.385956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:19.608853 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:19.608817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:19.714023 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:19.713987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:23.540138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:23.540103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:24.463987 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:24.463955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:28.612786 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:28.612750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:30.181539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:30.181503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:30.247282 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:30.247254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:49.191001 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.190921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:49.461718 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.461685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"]
Aug 05 17:46:49.462060 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.462048    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerName="main"
Aug 05 17:46:49.462104 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.462061    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerName="main"
Aug 05 17:46:49.462138 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.462115    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2019bfa-a1d4-4fbe-a96f-64535c2ac5ac" containerName="main"
Aug 05 17:46:49.465082 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.465065    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.475757 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.475736    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisvc-router-managed-test-llm-e45d1f79-epp-sa-dockercfg-mgz6n\""
Aug 05 17:46:49.477837 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.477821    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"kube-root-ca.crt\""
Aug 05 17:46:49.479108 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.479092    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Aug 05 17:46:49.479186 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.479144    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"openshift-service-ca.crt\""
Aug 05 17:46:49.479657 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.479635    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"default-dockercfg-9v97x\""
Aug 05 17:46:49.490105 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.490071    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"]
Aug 05 17:46:49.510244 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.510214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:46:49.535817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.535778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.535817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.535820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rfc4\" (UniqueName: \"kubernetes.io/projected/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-kube-api-access-8rfc4\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.637170 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.637133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rfc4\" (UniqueName: \"kubernetes.io/projected/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-kube-api-access-8rfc4\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.637402 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.637227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.639700 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.639670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.647017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.646981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rfc4\" (UniqueName: \"kubernetes.io/projected/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-kube-api-access-8rfc4\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.775178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.775091    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:49.913163 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:49.913118    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"]
Aug 05 17:46:49.916059 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:46:49.916033    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942c8f10_9ef9_4a77_afa1_a5a8ce3917a7.slice/crio-e0d164aa48bf11c105d2ea67402828dc4b3b32a7bf9dec4652f45e0b2f4bbf12 WatchSource:0}: Error finding container e0d164aa48bf11c105d2ea67402828dc4b3b32a7bf9dec4652f45e0b2f4bbf12: Status 404 returned error can't find the container with id e0d164aa48bf11c105d2ea67402828dc4b3b32a7bf9dec4652f45e0b2f4bbf12
Aug 05 17:46:50.727824 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:50.727785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" event={"ID":"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7","Type":"ContainerStarted","Data":"c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0"}
Aug 05 17:46:50.728288 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:50.727837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" event={"ID":"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7","Type":"ContainerStarted","Data":"e0d164aa48bf11c105d2ea67402828dc4b3b32a7bf9dec4652f45e0b2f4bbf12"}
Aug 05 17:46:50.728288 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:50.727860    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:46:50.746415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:46:50.746365    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" podStartSLOduration=1.746351798 podStartE2EDuration="1.746351798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:46:50.74510002 +0000 UTC m=+2069.278284266" watchObservedRunningTime="2026-08-05 17:46:50.746351798 +0000 UTC m=+2069.279536043"
Aug 05 17:47:01.830554 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:01.830520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:03.351783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:03.351751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:08.830970 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:08.830936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:16.936195 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:16.936160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:17.036351 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:17.036300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:21.210464 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:21.210435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:21.734392 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:21.734363    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:47:22.020016 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:22.019932    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:47:22.023441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:22.023416    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:47:22.024955 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:22.024928    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:47:22.028106 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:22.028088    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:47:48.913422 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:48.913383    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:48.919532 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:48.919497    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"]
Aug 05 17:47:48.979585 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:48.979552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:47:51.735476 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:47:51.735435    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:48:01.321211 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:01.321176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:01.734255 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:01.734217    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:48:03.938968 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:03.938913    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerName="main" containerID="cri-o://c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0" gracePeriod=45
Aug 05 17:48:03.970817 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:48:03.970789    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942c8f10_9ef9_4a77_afa1_a5a8ce3917a7.slice/crio-conmon-c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0.scope\": RecentStats: unable to find data in memory cache]"
Aug 05 17:48:03.970916 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:48:03.970885    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942c8f10_9ef9_4a77_afa1_a5a8ce3917a7.slice/crio-conmon-c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0.scope\": RecentStats: unable to find data in memory cache]"
Aug 05 17:48:04.065377 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.065351    2562 generic.go:356] "Generic (PLEG): container finished" podID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerID="c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0" exitCode=0
Aug 05 17:48:04.065489 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.065386    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" event={"ID":"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7","Type":"ContainerDied","Data":"c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0"}
Aug 05 17:48:04.179559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.179535    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:48:04.364234 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.364204    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-tls-certs\") pod \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") "
Aug 05 17:48:04.364234 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.364240    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rfc4\" (UniqueName: \"kubernetes.io/projected/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-kube-api-access-8rfc4\") pod \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\" (UID: \"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7\") "
Aug 05 17:48:04.366513 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.366480    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-kube-api-access-8rfc4" (OuterVolumeSpecName: "kube-api-access-8rfc4") pod "942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" (UID: "942c8f10-9ef9-4a77-afa1-a5a8ce3917a7"). InnerVolumeSpecName "kube-api-access-8rfc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:48:04.366627 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.366525    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" (UID: "942c8f10-9ef9-4a77-afa1-a5a8ce3917a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:48:04.465651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.465622    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:48:04.465651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:04.465648    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rfc4\" (UniqueName: \"kubernetes.io/projected/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7-kube-api-access-8rfc4\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:48:05.071152 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:05.071125    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"
Aug 05 17:48:05.071574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:05.071146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr" event={"ID":"942c8f10-9ef9-4a77-afa1-a5a8ce3917a7","Type":"ContainerDied","Data":"e0d164aa48bf11c105d2ea67402828dc4b3b32a7bf9dec4652f45e0b2f4bbf12"}
Aug 05 17:48:05.071574 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:05.071226    2562 scope.go:117] "RemoveContainer" containerID="c71a1db072fe2e68e082ac21d7602c1b98668c64e734cd9d9b6f8feb5b80a3f0"
Aug 05 17:48:05.094155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:05.094131    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"]
Aug 05 17:48:05.098243 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:05.098221    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheg8mcr"]
Aug 05 17:48:06.101551 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:06.101521    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" path="/var/lib/kubelet/pods/942c8f10-9ef9-4a77-afa1-a5a8ce3917a7/volumes"
Aug 05 17:48:08.370390 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:08.370360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:08.464265 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:08.464236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:14.779009 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:14.778927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:17.665718 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:17.665683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:17.769730 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:17.769696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:17.906698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:17.906668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:20.884080 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:20.884045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:20.986310 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:20.986278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:22.701399 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:22.701363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:30.608130 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:30.608057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:35.479765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.479734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:35.670736 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.670705    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"]
Aug 05 17:48:35.671071 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.671059    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerName="main"
Aug 05 17:48:35.671117 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.671072    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerName="main"
Aug 05 17:48:35.671163 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.671130    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="942c8f10-9ef9-4a77-afa1-a5a8ce3917a7" containerName="main"
Aug 05 17:48:35.673833 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.673813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.676617 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.676596    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-h6f25\""
Aug 05 17:48:35.677810 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.677788    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Aug 05 17:48:35.677903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.677794    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Aug 05 17:48:35.677903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.677825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Aug 05 17:48:35.678465 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.678382    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-router-managed-test-llm-5b1e8f15-epp-sa-dockercfg-lqldm\""
Aug 05 17:48:35.692057 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.692026    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"]
Aug 05 17:48:35.699510 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.699488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:35.702581 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.702551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfxj7\" (UniqueName: \"kubernetes.io/projected/5616eb0b-9790-43b5-b159-19df0972c7a2-kube-api-access-gfxj7\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.702710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.702692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5616eb0b-9790-43b5-b159-19df0972c7a2-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.803713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.803643    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5616eb0b-9790-43b5-b159-19df0972c7a2-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.803713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.803681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfxj7\" (UniqueName: \"kubernetes.io/projected/5616eb0b-9790-43b5-b159-19df0972c7a2-kube-api-access-gfxj7\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.806038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.806018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5616eb0b-9790-43b5-b159-19df0972c7a2-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.811947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.811924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfxj7\" (UniqueName: \"kubernetes.io/projected/5616eb0b-9790-43b5-b159-19df0972c7a2-kube-api-access-gfxj7\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:35.990459 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:35.990423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:36.121788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:36.121677    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"]
Aug 05 17:48:36.124671 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:48:36.124645    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5616eb0b_9790_43b5_b159_19df0972c7a2.slice/crio-c24399040a9332c8f8332219718a236ef78e704d40164a846ff0e47cb086d83b WatchSource:0}: Error finding container c24399040a9332c8f8332219718a236ef78e704d40164a846ff0e47cb086d83b: Status 404 returned error can't find the container with id c24399040a9332c8f8332219718a236ef78e704d40164a846ff0e47cb086d83b
Aug 05 17:48:36.211605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:36.211575    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" event={"ID":"5616eb0b-9790-43b5-b159-19df0972c7a2","Type":"ContainerStarted","Data":"e290473663cb2db3b208ae22b6faa73b5619ab6c04b4c16f5897162c19c22941"}
Aug 05 17:48:36.211760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:36.211641    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" event={"ID":"5616eb0b-9790-43b5-b159-19df0972c7a2","Type":"ContainerStarted","Data":"c24399040a9332c8f8332219718a236ef78e704d40164a846ff0e47cb086d83b"}
Aug 05 17:48:36.211857 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:36.211832    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:48:36.231380 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:36.231305    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" podStartSLOduration=1.231291258 podStartE2EDuration="1.231291258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:48:36.230585192 +0000 UTC m=+2174.763769438" watchObservedRunningTime="2026-08-05 17:48:36.231291258 +0000 UTC m=+2174.764475502"
Aug 05 17:48:37.849061 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:37.849030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:41.074808 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:41.074778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:47.345976 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:47.345944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:47.419563 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:47.419531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:48:52.431450 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:48:52.431402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:04.203155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:49:04.203108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:07.217784 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:49:07.217756    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:49:08.616280 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:49:08.616245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:08.673420 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:49:08.673389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:49:22.319222 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:49:22.319185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:04.420301 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.420268    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"]
Aug 05 17:50:04.423530 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.423512    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.426099 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.426082    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Aug 05 17:50:04.434300 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.434271    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"]
Aug 05 17:50:04.443015 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.442993    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:04.508564 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.508531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.508710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.508592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxrgf\" (UniqueName: \"kubernetes.io/projected/c4b5eccf-cdd5-4fde-ae38-75144582c993-kube-api-access-bxrgf\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.508710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.508657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.508710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.508690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.508710 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.508708    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b5eccf-cdd5-4fde-ae38-75144582c993-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.508846 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.508735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.609613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.609585    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.609765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.609649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxrgf\" (UniqueName: \"kubernetes.io/projected/c4b5eccf-cdd5-4fde-ae38-75144582c993-kube-api-access-bxrgf\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.609765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.609684    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.609765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.609715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.609765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.609739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b5eccf-cdd5-4fde-ae38-75144582c993-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.609765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.609765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.610166 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.610136    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.610166 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.610159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.610346 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.610174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.611919 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.611890    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.612121 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.612102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b5eccf-cdd5-4fde-ae38-75144582c993-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.622855 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.622829    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxrgf\" (UniqueName: \"kubernetes.io/projected/c4b5eccf-cdd5-4fde-ae38-75144582c993-kube-api-access-bxrgf\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.733962 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.733935    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:04.857701 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.857667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:04.863867 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:04.863828    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"]
Aug 05 17:50:04.866086 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:50:04.866054    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b5eccf_cdd5_4fde_ae38_75144582c993.slice/crio-43b386ece90d4691e1a2f0f3a710c23c0f43b4679198d9d7bb8373eca1ab5db9 WatchSource:0}: Error finding container 43b386ece90d4691e1a2f0f3a710c23c0f43b4679198d9d7bb8373eca1ab5db9: Status 404 returned error can't find the container with id 43b386ece90d4691e1a2f0f3a710c23c0f43b4679198d9d7bb8373eca1ab5db9
Aug 05 17:50:05.615941 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:05.615909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" event={"ID":"c4b5eccf-cdd5-4fde-ae38-75144582c993","Type":"ContainerStarted","Data":"46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e"}
Aug 05 17:50:05.616349 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:05.615958    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" event={"ID":"c4b5eccf-cdd5-4fde-ae38-75144582c993","Type":"ContainerStarted","Data":"43b386ece90d4691e1a2f0f3a710c23c0f43b4679198d9d7bb8373eca1ab5db9"}
Aug 05 17:50:05.638851 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:05.638783    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" podStartSLOduration=1.638769546 podStartE2EDuration="1.638769546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:05.637406229 +0000 UTC m=+2264.170590478" watchObservedRunningTime="2026-08-05 17:50:05.638769546 +0000 UTC m=+2264.171953791"
Aug 05 17:50:05.734768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:05.734742    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:14.734992 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:14.734959    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:14.747358 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:14.747332    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:15.675900 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:15.675867    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:37.215848 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:37.215814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:41.045644 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.045608    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"]
Aug 05 17:50:41.055378 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:50:41.055339    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:50:41.055378 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:50:41.055378 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:50:41.055378 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e" cmd=["/bin/sleep","15"]
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.055400    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" message="" err=<
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:50:41.055418    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerName="main" containerID="cri-o://46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e"
Aug 05 17:50:41.055713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.055462    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerName="main" containerID="cri-o://46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e" gracePeriod=60
Aug 05 17:50:41.056759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.056737    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:41.156550 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.156516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:41.665783 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.665744    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": dial tcp 10.132.0.54:8000: connect: connection refused"
Aug 05 17:50:41.786239 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.786211    2562 generic.go:356] "Generic (PLEG): container finished" podID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerID="46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e" exitCode=0
Aug 05 17:50:41.786406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.786283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" event={"ID":"c4b5eccf-cdd5-4fde-ae38-75144582c993","Type":"ContainerDied","Data":"46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e"}
Aug 05 17:50:41.802482 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.802459    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:41.919791 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919708    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-model-cache\") pod \"c4b5eccf-cdd5-4fde-ae38-75144582c993\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") "
Aug 05 17:50:41.919959 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919840    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b5eccf-cdd5-4fde-ae38-75144582c993-tls-certs\") pod \"c4b5eccf-cdd5-4fde-ae38-75144582c993\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") "
Aug 05 17:50:41.919959 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919874    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-home\") pod \"c4b5eccf-cdd5-4fde-ae38-75144582c993\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") "
Aug 05 17:50:41.919959 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919909    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxrgf\" (UniqueName: \"kubernetes.io/projected/c4b5eccf-cdd5-4fde-ae38-75144582c993-kube-api-access-bxrgf\") pod \"c4b5eccf-cdd5-4fde-ae38-75144582c993\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") "
Aug 05 17:50:41.920125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919960    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-dshm\") pod \"c4b5eccf-cdd5-4fde-ae38-75144582c993\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") "
Aug 05 17:50:41.920125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4b5eccf-cdd5-4fde-ae38-75144582c993" (UID: "c4b5eccf-cdd5-4fde-ae38-75144582c993"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:50:41.920125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.919986    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-tmp-dir\") pod \"c4b5eccf-cdd5-4fde-ae38-75144582c993\" (UID: \"c4b5eccf-cdd5-4fde-ae38-75144582c993\") "
Aug 05 17:50:41.920279 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.920197    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4b5eccf-cdd5-4fde-ae38-75144582c993" (UID: "c4b5eccf-cdd5-4fde-ae38-75144582c993"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:50:41.920359 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.920309    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:50:41.920359 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.920345    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:50:41.920478 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.920451    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-home" (OuterVolumeSpecName: "home") pod "c4b5eccf-cdd5-4fde-ae38-75144582c993" (UID: "c4b5eccf-cdd5-4fde-ae38-75144582c993"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:50:41.922728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.922675    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4b5eccf-cdd5-4fde-ae38-75144582c993-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4b5eccf-cdd5-4fde-ae38-75144582c993" (UID: "c4b5eccf-cdd5-4fde-ae38-75144582c993"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:50:41.922728 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.922692    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b5eccf-cdd5-4fde-ae38-75144582c993-kube-api-access-bxrgf" (OuterVolumeSpecName: "kube-api-access-bxrgf") pod "c4b5eccf-cdd5-4fde-ae38-75144582c993" (UID: "c4b5eccf-cdd5-4fde-ae38-75144582c993"). InnerVolumeSpecName "kube-api-access-bxrgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:50:41.924068 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:41.924038    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-dshm" (OuterVolumeSpecName: "dshm") pod "c4b5eccf-cdd5-4fde-ae38-75144582c993" (UID: "c4b5eccf-cdd5-4fde-ae38-75144582c993"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:50:42.021765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.021729    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b5eccf-cdd5-4fde-ae38-75144582c993-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:50:42.021765 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.021763    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:50:42.021970 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.021776    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxrgf\" (UniqueName: \"kubernetes.io/projected/c4b5eccf-cdd5-4fde-ae38-75144582c993-kube-api-access-bxrgf\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:50:42.021970 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.021791    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b5eccf-cdd5-4fde-ae38-75144582c993-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:50:42.792662 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.792623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk" event={"ID":"c4b5eccf-cdd5-4fde-ae38-75144582c993","Type":"ContainerDied","Data":"43b386ece90d4691e1a2f0f3a710c23c0f43b4679198d9d7bb8373eca1ab5db9"}
Aug 05 17:50:42.793124 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.792692    2562 scope.go:117] "RemoveContainer" containerID="46b7723b8a6a93c9c46a6be28d31591089e5e68cea24dd7134e3d05f4cc1b59e"
Aug 05 17:50:42.793124 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.792637    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"
Aug 05 17:50:42.815190 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.815161    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"]
Aug 05 17:50:42.819466 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:42.819444    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995kdbxk"]
Aug 05 17:50:44.102225 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:44.102190    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" path="/var/lib/kubelet/pods/c4b5eccf-cdd5-4fde-ae38-75144582c993/volumes"
Aug 05 17:50:46.383050 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:46.383002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:46.483655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:46.483618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:46.589505 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:46.589476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:47.943133 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:47.943096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:48.725030 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:48.725000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:51.437535 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:51.437503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:52.603168 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.603135    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"]
Aug 05 17:50:52.603547 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.603495    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerName="main"
Aug 05 17:50:52.603547 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.603507    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerName="main"
Aug 05 17:50:52.603626 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.603585    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b5eccf-cdd5-4fde-ae38-75144582c993" containerName="main"
Aug 05 17:50:52.607950 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.607926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.610505 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.610482    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"openshift-service-ca.crt\""
Aug 05 17:50:52.611624 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.611603    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"default-dockercfg-f25g4\""
Aug 05 17:50:52.611731 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.611661    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"kube-root-ca.crt\""
Aug 05 17:50:52.611795 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.611752    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-three-member-group-c33d828b\"/\"tri-v1-kserve-self-signed-certs\""
Aug 05 17:50:52.617835 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.617816    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"]
Aug 05 17:50:52.622428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.622406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:50:52.709545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.709505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-home\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.709545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.709544    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9452k\" (UniqueName: \"kubernetes.io/projected/684710be-5217-4b46-b3ed-399fedaee377-kube-api-access-9452k\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.709760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.709571    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-dshm\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.709760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.709632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-tmp-dir\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.709760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.709704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/684710be-5217-4b46-b3ed-399fedaee377-tls-certs\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.709760 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.709744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-model-cache\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810385 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-dshm\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-tmp-dir\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/684710be-5217-4b46-b3ed-399fedaee377-tls-certs\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-model-cache\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-home\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810778 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9452k\" (UniqueName: \"kubernetes.io/projected/684710be-5217-4b46-b3ed-399fedaee377-kube-api-access-9452k\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-tmp-dir\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810942 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810871    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-model-cache\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.810942 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.810919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-home\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.813182 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.813161    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-dshm\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.813406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.813390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/684710be-5217-4b46-b3ed-399fedaee377-tls-certs\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.830278 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.830254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9452k\" (UniqueName: \"kubernetes.io/projected/684710be-5217-4b46-b3ed-399fedaee377-kube-api-access-9452k\") pod \"tri-v1-kserve-79f8f98bc5-59rrj\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") " pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:52.919309 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:52.919235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:53.068541 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:53.068506    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"]
Aug 05 17:50:53.070365 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:50:53.070335    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684710be_5217_4b46_b3ed_399fedaee377.slice/crio-73a3889b4092876e963df53f9caa734a1713672c4a1cd3954f247c5559e4daa2 WatchSource:0}: Error finding container 73a3889b4092876e963df53f9caa734a1713672c4a1cd3954f247c5559e4daa2: Status 404 returned error can't find the container with id 73a3889b4092876e963df53f9caa734a1713672c4a1cd3954f247c5559e4daa2
Aug 05 17:50:53.846136 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:53.846090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" event={"ID":"684710be-5217-4b46-b3ed-399fedaee377","Type":"ContainerStarted","Data":"0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db"}
Aug 05 17:50:53.846607 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:53.846184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" event={"ID":"684710be-5217-4b46-b3ed-399fedaee377","Type":"ContainerStarted","Data":"73a3889b4092876e963df53f9caa734a1713672c4a1cd3954f247c5559e4daa2"}
Aug 05 17:50:53.865907 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:53.865856    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" podStartSLOduration=1.86584189 podStartE2EDuration="1.86584189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:53.865213268 +0000 UTC m=+2312.398397512" watchObservedRunningTime="2026-08-05 17:50:53.86584189 +0000 UTC m=+2312.399026135"
Aug 05 17:50:53.919553 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:53.919514    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:50:56.224605 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:50:56.224568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:01.024493 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:01.024453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:02.808736 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:02.808704    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"]
Aug 05 17:51:02.817066 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:02.817038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:02.878554 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:02.878526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:02.920462 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:02.920439    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:51:02.932769 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:02.932748    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:51:03.902651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:03.902621    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:51:07.218006 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:07.217968    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:51:17.217936 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:17.217850    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:51:17.828701 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:17.828643    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerName="main" containerID="cri-o://e290473663cb2db3b208ae22b6faa73b5619ab6c04b4c16f5897162c19c22941" gracePeriod=45
Aug 05 17:51:17.928567 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:17.928534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:17.961498 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:17.961469    2562 generic.go:356] "Generic (PLEG): container finished" podID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerID="e290473663cb2db3b208ae22b6faa73b5619ab6c04b4c16f5897162c19c22941" exitCode=0
Aug 05 17:51:17.961651 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:17.961544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" event={"ID":"5616eb0b-9790-43b5-b159-19df0972c7a2","Type":"ContainerDied","Data":"e290473663cb2db3b208ae22b6faa73b5619ab6c04b4c16f5897162c19c22941"}
Aug 05 17:51:18.073657 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.073628    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:51:18.227236 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.227210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfxj7\" (UniqueName: \"kubernetes.io/projected/5616eb0b-9790-43b5-b159-19df0972c7a2-kube-api-access-gfxj7\") pod \"5616eb0b-9790-43b5-b159-19df0972c7a2\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") "
Aug 05 17:51:18.227715 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.227263    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5616eb0b-9790-43b5-b159-19df0972c7a2-tls-certs\") pod \"5616eb0b-9790-43b5-b159-19df0972c7a2\" (UID: \"5616eb0b-9790-43b5-b159-19df0972c7a2\") "
Aug 05 17:51:18.229250 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.229227    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5616eb0b-9790-43b5-b159-19df0972c7a2-kube-api-access-gfxj7" (OuterVolumeSpecName: "kube-api-access-gfxj7") pod "5616eb0b-9790-43b5-b159-19df0972c7a2" (UID: "5616eb0b-9790-43b5-b159-19df0972c7a2"). InnerVolumeSpecName "kube-api-access-gfxj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:51:18.229381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.229286    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5616eb0b-9790-43b5-b159-19df0972c7a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5616eb0b-9790-43b5-b159-19df0972c7a2" (UID: "5616eb0b-9790-43b5-b159-19df0972c7a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:51:18.328361 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.328300    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfxj7\" (UniqueName: \"kubernetes.io/projected/5616eb0b-9790-43b5-b159-19df0972c7a2-kube-api-access-gfxj7\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:51:18.328361 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.328353    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5616eb0b-9790-43b5-b159-19df0972c7a2-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:51:18.966959 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.966926    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"
Aug 05 17:51:18.967186 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.966956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62" event={"ID":"5616eb0b-9790-43b5-b159-19df0972c7a2","Type":"ContainerDied","Data":"c24399040a9332c8f8332219718a236ef78e704d40164a846ff0e47cb086d83b"}
Aug 05 17:51:18.967186 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.967014    2562 scope.go:117] "RemoveContainer" containerID="e290473663cb2db3b208ae22b6faa73b5619ab6c04b4c16f5897162c19c22941"
Aug 05 17:51:18.988782 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.988752    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"]
Aug 05 17:51:18.990275 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:18.990253    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-schengw62"]
Aug 05 17:51:19.838491 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:19.838461    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:19.934861 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:19.934832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:20.102880 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:20.102805    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" path="/var/lib/kubelet/pods/5616eb0b-9790-43b5-b159-19df0972c7a2/volumes"
Aug 05 17:51:25.803079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:25.803049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:26.446024 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:26.445971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:28.693402 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:28.693364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:32.235325 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:32.235269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:32.291600 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:32.291567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:34.340913 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:34.340884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:34.457862 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:34.457829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:58.310687 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.310649    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"]
Aug 05 17:51:58.311226 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.311086    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerName="main"
Aug 05 17:51:58.311226 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.311101    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerName="main"
Aug 05 17:51:58.311226 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.311167    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5616eb0b-9790-43b5-b159-19df0972c7a2" containerName="main"
Aug 05 17:51:58.314331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.314300    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.316874 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.316850    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Aug 05 17:51:58.317905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.317884    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-54kk4\""
Aug 05 17:51:58.317969 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.317907    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Aug 05 17:51:58.318027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.317908    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Aug 05 17:51:58.324845 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.324812    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"]
Aug 05 17:51:58.333746 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.333722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:58.469147 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.469117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.469370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.469154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.469370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.469180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gc5p\" (UniqueName: \"kubernetes.io/projected/4d6686b6-4869-43a2-bf21-9d4e727d1b30-kube-api-access-2gc5p\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.469370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.469235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.469370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.469267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.469562 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.469388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.562053 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.561972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:51:58.570107 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570188    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570274 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570393 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570454 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gc5p\" (UniqueName: \"kubernetes.io/projected/4d6686b6-4869-43a2-bf21-9d4e727d1b30-kube-api-access-2gc5p\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570522 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570587 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.570713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.570693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.572462 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.572444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.572830 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.572813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.579118 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.579094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gc5p\" (UniqueName: \"kubernetes.io/projected/4d6686b6-4869-43a2-bf21-9d4e727d1b30-kube-api-access-2gc5p\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.625203 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.625166    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:51:58.755037 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:58.755006    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"]
Aug 05 17:51:58.757506 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:51:58.757478    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6686b6_4869_43a2_bf21_9d4e727d1b30.slice/crio-55e5f586bb12c8e3dc784cd567ad1d0aa9a050f8a333df1a47af5dacd5ba01db WatchSource:0}: Error finding container 55e5f586bb12c8e3dc784cd567ad1d0aa9a050f8a333df1a47af5dacd5ba01db: Status 404 returned error can't find the container with id 55e5f586bb12c8e3dc784cd567ad1d0aa9a050f8a333df1a47af5dacd5ba01db
Aug 05 17:51:59.161643 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:59.161606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" event={"ID":"4d6686b6-4869-43a2-bf21-9d4e727d1b30","Type":"ContainerStarted","Data":"8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1"}
Aug 05 17:51:59.161824 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:59.161660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" event={"ID":"4d6686b6-4869-43a2-bf21-9d4e727d1b30","Type":"ContainerStarted","Data":"55e5f586bb12c8e3dc784cd567ad1d0aa9a050f8a333df1a47af5dacd5ba01db"}
Aug 05 17:51:59.181078 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:59.181033    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" podStartSLOduration=1.181018247 podStartE2EDuration="1.181018247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:51:59.179353458 +0000 UTC m=+2377.712537705" watchObservedRunningTime="2026-08-05 17:51:59.181018247 +0000 UTC m=+2377.714202543"
Aug 05 17:51:59.625917 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:51:59.625882    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:52:08.625814 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:08.625779    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:52:08.638140 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:08.638114    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:52:09.218075 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:09.218046    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:52:10.221043 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:10.221008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:11.814734 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:11.814701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:13.178609 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:13.178573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:22.049341 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:22.049290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:52:22.053518 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:22.053497    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:52:22.053655 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:22.053640    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:52:22.057903 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:22.057884    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:52:36.813981 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:36.813951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:36.917253 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:36.917226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:37.014370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:37.014335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:41.787816 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:41.787777    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"]
Aug 05 17:52:41.861415 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:41.861374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:45.997616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:45.997539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:46.238905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.238874    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"]
Aug 05 17:52:46.242817 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.242797    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.246428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.246402    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Aug 05 17:52:46.246559 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.246448    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa-dockercfg-psxtf\""
Aug 05 17:52:46.252586 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.252562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"]
Aug 05 17:52:46.277403 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.277365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:46.390780 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.390752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.390780 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.390783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h798s\" (UniqueName: \"kubernetes.io/projected/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-kube-api-access-h798s\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.491614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.491569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.491614 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.491614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h798s\" (UniqueName: \"kubernetes.io/projected/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-kube-api-access-h798s\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.494009 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.493988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.499434 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.499410    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h798s\" (UniqueName: \"kubernetes.io/projected/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-kube-api-access-h798s\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.554127 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.554065    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:46.685887 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:46.685862    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"]
Aug 05 17:52:46.688102 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:52:46.688074    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7a8ebc_2fe4_480a_a7b0_deda08eaaee7.slice/crio-a18b007a8d4e3036310ebf8215298fb2c946b8fa40e16ffebe060737bfb16024 WatchSource:0}: Error finding container a18b007a8d4e3036310ebf8215298fb2c946b8fa40e16ffebe060737bfb16024: Status 404 returned error can't find the container with id a18b007a8d4e3036310ebf8215298fb2c946b8fa40e16ffebe060737bfb16024
Aug 05 17:52:47.395096 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:47.395055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" event={"ID":"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7","Type":"ContainerStarted","Data":"9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9"}
Aug 05 17:52:47.395539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:47.395118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" event={"ID":"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7","Type":"ContainerStarted","Data":"a18b007a8d4e3036310ebf8215298fb2c946b8fa40e16ffebe060737bfb16024"}
Aug 05 17:52:47.395539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:47.395167    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:52:47.414488 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:47.414443    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" podStartSLOduration=1.4144290449999999 podStartE2EDuration="1.414429045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:52:47.412411077 +0000 UTC m=+2425.945595323" watchObservedRunningTime="2026-08-05 17:52:47.414429045 +0000 UTC m=+2425.947613289"
Aug 05 17:52:49.940055 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:49.940014    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:50.223579 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:50.223544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:51.310201 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:51.310169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:52:56.807571 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:56.807529    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" podUID="684710be-5217-4b46-b3ed-399fedaee377" containerName="main" containerID="cri-o://0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db" gracePeriod=45
Aug 05 17:52:56.891876 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:56.891835    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" podUID="684710be-5217-4b46-b3ed-399fedaee377" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": dial tcp 10.132.0.55:8000: connect: connection refused"
Aug 05 17:52:57.053723 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.053699    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:52:57.182112 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182016    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-tmp-dir\") pod \"684710be-5217-4b46-b3ed-399fedaee377\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") "
Aug 05 17:52:57.182112 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182097    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-home\") pod \"684710be-5217-4b46-b3ed-399fedaee377\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") "
Aug 05 17:52:57.182349 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182138    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-dshm\") pod \"684710be-5217-4b46-b3ed-399fedaee377\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") "
Aug 05 17:52:57.182349 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182170    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-model-cache\") pod \"684710be-5217-4b46-b3ed-399fedaee377\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") "
Aug 05 17:52:57.182349 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182216    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9452k\" (UniqueName: \"kubernetes.io/projected/684710be-5217-4b46-b3ed-399fedaee377-kube-api-access-9452k\") pod \"684710be-5217-4b46-b3ed-399fedaee377\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") "
Aug 05 17:52:57.182349 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182247    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/684710be-5217-4b46-b3ed-399fedaee377-tls-certs\") pod \"684710be-5217-4b46-b3ed-399fedaee377\" (UID: \"684710be-5217-4b46-b3ed-399fedaee377\") "
Aug 05 17:52:57.182349 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182304    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "684710be-5217-4b46-b3ed-399fedaee377" (UID: "684710be-5217-4b46-b3ed-399fedaee377"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:52:57.182608 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182494    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-home" (OuterVolumeSpecName: "home") pod "684710be-5217-4b46-b3ed-399fedaee377" (UID: "684710be-5217-4b46-b3ed-399fedaee377"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:52:57.182608 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182529    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-model-cache" (OuterVolumeSpecName: "model-cache") pod "684710be-5217-4b46-b3ed-399fedaee377" (UID: "684710be-5217-4b46-b3ed-399fedaee377"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:52:57.182608 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182603    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:52:57.182740 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.182623    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:52:57.184341 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.184308    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-dshm" (OuterVolumeSpecName: "dshm") pod "684710be-5217-4b46-b3ed-399fedaee377" (UID: "684710be-5217-4b46-b3ed-399fedaee377"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:52:57.184441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.184374    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/684710be-5217-4b46-b3ed-399fedaee377-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "684710be-5217-4b46-b3ed-399fedaee377" (UID: "684710be-5217-4b46-b3ed-399fedaee377"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:52:57.184441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.184396    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/684710be-5217-4b46-b3ed-399fedaee377-kube-api-access-9452k" (OuterVolumeSpecName: "kube-api-access-9452k") pod "684710be-5217-4b46-b3ed-399fedaee377" (UID: "684710be-5217-4b46-b3ed-399fedaee377"). InnerVolumeSpecName "kube-api-access-9452k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:52:57.283307 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.283280    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:52:57.283307 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.283304    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/684710be-5217-4b46-b3ed-399fedaee377-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:52:57.283489 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.283331    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9452k\" (UniqueName: \"kubernetes.io/projected/684710be-5217-4b46-b3ed-399fedaee377-kube-api-access-9452k\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:52:57.283489 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.283341    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/684710be-5217-4b46-b3ed-399fedaee377-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:52:57.444167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.444137    2562 generic.go:356] "Generic (PLEG): container finished" podID="684710be-5217-4b46-b3ed-399fedaee377" containerID="0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db" exitCode=0
Aug 05 17:52:57.444306 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.444178    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"
Aug 05 17:52:57.444306 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.444213    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" event={"ID":"684710be-5217-4b46-b3ed-399fedaee377","Type":"ContainerDied","Data":"0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db"}
Aug 05 17:52:57.444306 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.444269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj" event={"ID":"684710be-5217-4b46-b3ed-399fedaee377","Type":"ContainerDied","Data":"73a3889b4092876e963df53f9caa734a1713672c4a1cd3954f247c5559e4daa2"}
Aug 05 17:52:57.444306 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.444292    2562 scope.go:117] "RemoveContainer" containerID="0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db"
Aug 05 17:52:57.453187 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.453169    2562 scope.go:117] "RemoveContainer" containerID="0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db"
Aug 05 17:52:57.453470 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:52:57.453449    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db\": container with ID starting with 0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db not found: ID does not exist" containerID="0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db"
Aug 05 17:52:57.453529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.453478    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db"} err="failed to get container status \"0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db\": rpc error: code = NotFound desc = could not find container \"0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db\": container with ID starting with 0fd50580b4a375cdc39a3f1c02a0b32afb67926dbdd9227ea1227bfedf67e4db not found: ID does not exist"
Aug 05 17:52:57.466295 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.466274    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"]
Aug 05 17:52:57.471905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:57.471875    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-three-member-group-c33d828b/tri-v1-kserve-79f8f98bc5-59rrj"]
Aug 05 17:52:58.102360 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:58.102307    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="684710be-5217-4b46-b3ed-399fedaee377" path="/var/lib/kubelet/pods/684710be-5217-4b46-b3ed-399fedaee377/volumes"
Aug 05 17:52:59.820526 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:52:59.820490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:11.123532 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:11.123499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:18.401961 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:18.401932    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:53:19.444742 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:19.444698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:19.842589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:19.842512    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"]
Aug 05 17:53:19.847027 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:19.846981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:19.904280 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:19.904246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:25.419599 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:25.419557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:25.477381 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:25.477355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:27.510921 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:27.510889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:28.402406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:28.402368    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" podUID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:53:31.632881 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:31.632838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:34.861911 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:34.861872    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" podUID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" containerName="main" containerID="cri-o://9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9" gracePeriod=45
Aug 05 17:53:35.113663 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.113608    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:53:35.208211 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.208176    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-tls-certs\") pod \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") "
Aug 05 17:53:35.208411 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.208334    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h798s\" (UniqueName: \"kubernetes.io/projected/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-kube-api-access-h798s\") pod \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\" (UID: \"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7\") "
Aug 05 17:53:35.210497 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.210470    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" (UID: "fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:53:35.210593 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.210509    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-kube-api-access-h798s" (OuterVolumeSpecName: "kube-api-access-h798s") pod "fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" (UID: "fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7"). InnerVolumeSpecName "kube-api-access-h798s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:53:35.309716 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.309685    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h798s\" (UniqueName: \"kubernetes.io/projected/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-kube-api-access-h798s\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:35.309716 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.309711    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:35.631788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.625769    2562 generic.go:356] "Generic (PLEG): container finished" podID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" containerID="9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9" exitCode=0
Aug 05 17:53:35.631788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.625979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" event={"ID":"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7","Type":"ContainerDied","Data":"9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9"}
Aug 05 17:53:35.631788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.626063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8" event={"ID":"fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7","Type":"ContainerDied","Data":"a18b007a8d4e3036310ebf8215298fb2c946b8fa40e16ffebe060737bfb16024"}
Aug 05 17:53:35.631788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.626107    2562 scope.go:117] "RemoveContainer" containerID="9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9"
Aug 05 17:53:35.631788 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.626348    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"
Aug 05 17:53:35.641528 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.641510    2562 scope.go:117] "RemoveContainer" containerID="9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9"
Aug 05 17:53:35.641809 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:53:35.641782    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9\": container with ID starting with 9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9 not found: ID does not exist" containerID="9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9"
Aug 05 17:53:35.641868 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.641817    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9"} err="failed to get container status \"9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9\": rpc error: code = NotFound desc = could not find container \"9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9\": container with ID starting with 9a0a84edd46a788a37fab575b9a4dc4c765f7263e4b87980737616bcdd8cd6d9 not found: ID does not exist"
Aug 05 17:53:35.656051 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.656015    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"]
Aug 05 17:53:35.657910 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:35.657889    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheqhbf8"]
Aug 05 17:53:36.102662 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:36.102631    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" path="/var/lib/kubelet/pods/fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7/volumes"
Aug 05 17:53:47.344310 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.344275    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"]
Aug 05 17:53:47.351824 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.351797    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:47.354707 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:53:47.354677    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:53:47.354707 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:53:47.354707 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:53:47.354707 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1" cmd=["/bin/sleep","15"]
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.354731    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" message="" err=<
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:53:47.354747    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" podUID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" containerName="main" containerID="cri-o://8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1"
Aug 05 17:53:47.355008 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.354793    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" podUID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" containerName="main" containerID="cri-o://8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1" gracePeriod=60
Aug 05 17:53:47.411178 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.411141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:53:47.686199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.686119    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" containerID="8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1" exitCode=0
Aug 05 17:53:47.686360 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:47.686193    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" event={"ID":"4d6686b6-4869-43a2-bf21-9d4e727d1b30","Type":"ContainerDied","Data":"8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1"}
Aug 05 17:53:48.097991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.097969    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:53:48.219044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.218968    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tmp-dir\") pod \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") "
Aug 05 17:53:48.219044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219026    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-model-cache\") pod \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") "
Aug 05 17:53:48.219245 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219141    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tls-certs\") pod \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") "
Aug 05 17:53:48.219245 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219208    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4d6686b6-4869-43a2-bf21-9d4e727d1b30" (UID: "4d6686b6-4869-43a2-bf21-9d4e727d1b30"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:48.219245 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219233    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-dshm\") pod \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") "
Aug 05 17:53:48.219441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219261    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-home\") pod \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") "
Aug 05 17:53:48.219441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219339    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-model-cache" (OuterVolumeSpecName: "model-cache") pod "4d6686b6-4869-43a2-bf21-9d4e727d1b30" (UID: "4d6686b6-4869-43a2-bf21-9d4e727d1b30"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:48.219441 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219360    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gc5p\" (UniqueName: \"kubernetes.io/projected/4d6686b6-4869-43a2-bf21-9d4e727d1b30-kube-api-access-2gc5p\") pod \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\" (UID: \"4d6686b6-4869-43a2-bf21-9d4e727d1b30\") "
Aug 05 17:53:48.219550 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219531    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-home" (OuterVolumeSpecName: "home") pod "4d6686b6-4869-43a2-bf21-9d4e727d1b30" (UID: "4d6686b6-4869-43a2-bf21-9d4e727d1b30"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:48.219633 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219617    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:48.219688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219641    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:48.219688 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.219653    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:48.221175 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.221146    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4d6686b6-4869-43a2-bf21-9d4e727d1b30" (UID: "4d6686b6-4869-43a2-bf21-9d4e727d1b30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:53:48.221273 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.221184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-dshm" (OuterVolumeSpecName: "dshm") pod "4d6686b6-4869-43a2-bf21-9d4e727d1b30" (UID: "4d6686b6-4869-43a2-bf21-9d4e727d1b30"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:48.221335 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.221290    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6686b6-4869-43a2-bf21-9d4e727d1b30-kube-api-access-2gc5p" (OuterVolumeSpecName: "kube-api-access-2gc5p") pod "4d6686b6-4869-43a2-bf21-9d4e727d1b30" (UID: "4d6686b6-4869-43a2-bf21-9d4e727d1b30"). InnerVolumeSpecName "kube-api-access-2gc5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:53:48.319983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.319949    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gc5p\" (UniqueName: \"kubernetes.io/projected/4d6686b6-4869-43a2-bf21-9d4e727d1b30-kube-api-access-2gc5p\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:48.319983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.319985    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6686b6-4869-43a2-bf21-9d4e727d1b30-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:48.320213 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.319996    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6686b6-4869-43a2-bf21-9d4e727d1b30-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:53:48.691616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.691533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd" event={"ID":"4d6686b6-4869-43a2-bf21-9d4e727d1b30","Type":"ContainerDied","Data":"55e5f586bb12c8e3dc784cd567ad1d0aa9a050f8a333df1a47af5dacd5ba01db"}
Aug 05 17:53:48.691616 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.691584    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"
Aug 05 17:53:48.692092 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.691623    2562 scope.go:117] "RemoveContainer" containerID="8d28b7c73bca2a4a13858c42019f10e59e97133073047ec67b39f05e996affb1"
Aug 05 17:53:48.712800 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.712772    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"]
Aug 05 17:53:48.716691 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:48.716668    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969zrmfd"]
Aug 05 17:53:50.102071 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:53:50.102036    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" path="/var/lib/kubelet/pods/4d6686b6-4869-43a2-bf21-9d4e727d1b30/volumes"
Aug 05 17:54:04.655899 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:04.655866    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:04.744238 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:04.744209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:11.192994 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:11.192957    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:14.650762 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:14.650688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:14.740548 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:14.740521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:14.838758 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:14.838730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:17.330844 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:17.330806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:17.390170 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:17.390138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:18.430976 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.430939    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:18.604690 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.604659    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"]
Aug 05 17:54:18.605038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605016    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684710be-5217-4b46-b3ed-399fedaee377" containerName="main"
Aug 05 17:54:18.605038 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605033    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="684710be-5217-4b46-b3ed-399fedaee377" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605053    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605059    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605068    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605074    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605136    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="684710be-5217-4b46-b3ed-399fedaee377" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605143    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7a8ebc-2fe4-480a-a7b0-deda08eaaee7" containerName="main"
Aug 05 17:54:18.605199 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.605150    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d6686b6-4869-43a2-bf21-9d4e727d1b30" containerName="main"
Aug 05 17:54:18.608189 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.608170    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.612241 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.612221    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa-dockercfg-rkqqt\""
Aug 05 17:54:18.612391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.612246    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\""
Aug 05 17:54:18.612391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.612252    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"openshift-service-ca.crt\""
Aug 05 17:54:18.612391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.612294    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"kube-root-ca.crt\""
Aug 05 17:54:18.612391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.612221    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"default-dockercfg-lcqqf\""
Aug 05 17:54:18.616536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.616517    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"]
Aug 05 17:54:18.640953 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.640925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:18.659061 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.659038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xs6q\" (UniqueName: \"kubernetes.io/projected/88f52adf-8875-4e5e-9606-86bc93298a27-kube-api-access-6xs6q\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.659173 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.659096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f52adf-8875-4e5e-9606-86bc93298a27-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.759873 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.759849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f52adf-8875-4e5e-9606-86bc93298a27-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.759981 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.759930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xs6q\" (UniqueName: \"kubernetes.io/projected/88f52adf-8875-4e5e-9606-86bc93298a27-kube-api-access-6xs6q\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.762421 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.762394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f52adf-8875-4e5e-9606-86bc93298a27-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.768163 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.768142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xs6q\" (UniqueName: \"kubernetes.io/projected/88f52adf-8875-4e5e-9606-86bc93298a27-kube-api-access-6xs6q\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:18.919156 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:18.919124    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:19.048366 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:19.048342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"]
Aug 05 17:54:19.050926 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:54:19.050902    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f52adf_8875_4e5e_9606_86bc93298a27.slice/crio-2de32f2bcd82f15369de59c06c745940d088576939e18b9d18c0acb61f5f238f WatchSource:0}: Error finding container 2de32f2bcd82f15369de59c06c745940d088576939e18b9d18c0acb61f5f238f: Status 404 returned error can't find the container with id 2de32f2bcd82f15369de59c06c745940d088576939e18b9d18c0acb61f5f238f
Aug 05 17:54:19.703004 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:19.702954    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:19.836341 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:19.836286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" event={"ID":"88f52adf-8875-4e5e-9606-86bc93298a27","Type":"ContainerStarted","Data":"d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f"}
Aug 05 17:54:19.836529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:19.836360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" event={"ID":"88f52adf-8875-4e5e-9606-86bc93298a27","Type":"ContainerStarted","Data":"2de32f2bcd82f15369de59c06c745940d088576939e18b9d18c0acb61f5f238f"}
Aug 05 17:54:19.836529 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:19.836445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:19.854813 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:19.854768    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" podStartSLOduration=1.8547537109999999 podStartE2EDuration="1.854753711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:19.853381051 +0000 UTC m=+2518.386565296" watchObservedRunningTime="2026-08-05 17:54:19.854753711 +0000 UTC m=+2518.387937955"
Aug 05 17:54:21.006768 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:21.006728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:21.809680 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:21.809635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:24.542047 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:24.542009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:24.641873 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:24.641839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:35.200155 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:35.200115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:44.242583 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:44.242551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:45.965464 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:45.965430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:50.842711 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:50.842678    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:54:55.150406 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:55.150374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:57.326421 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:57.326384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:54:57.426375 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:54:57.426344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:03.483671 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:03.483638    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:03.983117 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:03.983086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:21.202085 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:21.202054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:37.409250 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:37.409214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:37.486565 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:37.486531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:37.540553 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:37.540521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:42.485418 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:42.485385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:44.138531 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:44.138498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:55:57.541330 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:55:57.541287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:19.459046 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:19.459014    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:24.727463 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:24.727430    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"]
Aug 05 17:56:24.733864 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:24.733829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:24.834713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:24.834678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:25.775982 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:25.775944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:25.832668 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:25.832637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:30.842918 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:30.842882    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" podUID="88f52adf-8875-4e5e-9606-86bc93298a27" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:56:39.748489 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:39.748427    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" podUID="88f52adf-8875-4e5e-9606-86bc93298a27" containerName="main" containerID="cri-o://d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f" gracePeriod=45
Aug 05 17:56:40.003362 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.003282    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:56:40.114764 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.114736    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xs6q\" (UniqueName: \"kubernetes.io/projected/88f52adf-8875-4e5e-9606-86bc93298a27-kube-api-access-6xs6q\") pod \"88f52adf-8875-4e5e-9606-86bc93298a27\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") "
Aug 05 17:56:40.114946 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.114776    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f52adf-8875-4e5e-9606-86bc93298a27-tls-certs\") pod \"88f52adf-8875-4e5e-9606-86bc93298a27\" (UID: \"88f52adf-8875-4e5e-9606-86bc93298a27\") "
Aug 05 17:56:40.117079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.117053    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f52adf-8875-4e5e-9606-86bc93298a27-kube-api-access-6xs6q" (OuterVolumeSpecName: "kube-api-access-6xs6q") pod "88f52adf-8875-4e5e-9606-86bc93298a27" (UID: "88f52adf-8875-4e5e-9606-86bc93298a27"). InnerVolumeSpecName "kube-api-access-6xs6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:56:40.117188 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.117097    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f52adf-8875-4e5e-9606-86bc93298a27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88f52adf-8875-4e5e-9606-86bc93298a27" (UID: "88f52adf-8875-4e5e-9606-86bc93298a27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:56:40.216103 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.216061    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xs6q\" (UniqueName: \"kubernetes.io/projected/88f52adf-8875-4e5e-9606-86bc93298a27-kube-api-access-6xs6q\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:56:40.216103 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.216102    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f52adf-8875-4e5e-9606-86bc93298a27-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:56:40.477984 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.477949    2562 generic.go:356] "Generic (PLEG): container finished" podID="88f52adf-8875-4e5e-9606-86bc93298a27" containerID="d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f" exitCode=0
Aug 05 17:56:40.478167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.477997    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"
Aug 05 17:56:40.478167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.478012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" event={"ID":"88f52adf-8875-4e5e-9606-86bc93298a27","Type":"ContainerDied","Data":"d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f"}
Aug 05 17:56:40.478167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.478063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d" event={"ID":"88f52adf-8875-4e5e-9606-86bc93298a27","Type":"ContainerDied","Data":"2de32f2bcd82f15369de59c06c745940d088576939e18b9d18c0acb61f5f238f"}
Aug 05 17:56:40.478167 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.478092    2562 scope.go:117] "RemoveContainer" containerID="d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f"
Aug 05 17:56:40.486767 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.486748    2562 scope.go:117] "RemoveContainer" containerID="d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f"
Aug 05 17:56:40.487020 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:56:40.486996    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f\": container with ID starting with d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f not found: ID does not exist" containerID="d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f"
Aug 05 17:56:40.487080 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.487033    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f"} err="failed to get container status \"d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f\": rpc error: code = NotFound desc = could not find container \"d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f\": container with ID starting with d7ac252d5ee044138f657c8840f11b42a32e76d9de89461de3c92d273261182f not found: ID does not exist"
Aug 05 17:56:40.499106 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.499079    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"]
Aug 05 17:56:40.502475 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:40.502453    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheldd6d"]
Aug 05 17:56:42.102902 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:42.102867    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88f52adf-8875-4e5e-9606-86bc93298a27" path="/var/lib/kubelet/pods/88f52adf-8875-4e5e-9606-86bc93298a27/volumes"
Aug 05 17:56:44.751540 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:44.751496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:44.851079 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:44.851048    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:50.835718 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:50.835684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:56.767923 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:56.767851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:56.836490 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:56.836446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:58.004786 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.004759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:58.162073 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.162042    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"]
Aug 05 17:56:58.162434 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.162411    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f52adf-8875-4e5e-9606-86bc93298a27" containerName="main"
Aug 05 17:56:58.162434 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.162428    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f52adf-8875-4e5e-9606-86bc93298a27" containerName="main"
Aug 05 17:56:58.162594 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.162495    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f52adf-8875-4e5e-9606-86bc93298a27" containerName="main"
Aug 05 17:56:58.167101 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.167083    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.171253 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.171221    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"kube-root-ca.crt\""
Aug 05 17:56:58.171253 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.171227    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Aug 05 17:56:58.171458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.171263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"default-dockercfg-cpwzs\""
Aug 05 17:56:58.171458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.171245    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-xmkh7\""
Aug 05 17:56:58.171458 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.171297    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"openshift-service-ca.crt\""
Aug 05 17:56:58.181511 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.176869    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"]
Aug 05 17:56:58.191151 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.191128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:58.254538 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.254508    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:58.264177 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.264115    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nwwt\" (UniqueName: \"kubernetes.io/projected/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-kube-api-access-4nwwt\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.264177 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.264152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.355308 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.355273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:58.365496 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.365468    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nwwt\" (UniqueName: \"kubernetes.io/projected/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-kube-api-access-4nwwt\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.365590 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.365504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.367968 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.367943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.376445 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.376415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nwwt\" (UniqueName: \"kubernetes.io/projected/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-kube-api-access-4nwwt\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.455797 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.455762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:56:58.482938 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.482892    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:58.616664 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:58.616630    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"]
Aug 05 17:56:58.619655 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:56:58.619623    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cfb912_8a4d_43f6_b1d8_961b3f02f1ba.slice/crio-c4b82181a04c586babb8eb07f3a772c88873b5b668bc439132c6b2b2a42b9dea WatchSource:0}: Error finding container c4b82181a04c586babb8eb07f3a772c88873b5b668bc439132c6b2b2a42b9dea: Status 404 returned error can't find the container with id c4b82181a04c586babb8eb07f3a772c88873b5b668bc439132c6b2b2a42b9dea
Aug 05 17:56:59.569653 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:59.569605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" event={"ID":"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba","Type":"ContainerStarted","Data":"aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af"}
Aug 05 17:56:59.570137 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:59.569684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" event={"ID":"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba","Type":"ContainerStarted","Data":"c4b82181a04c586babb8eb07f3a772c88873b5b668bc439132c6b2b2a42b9dea"}
Aug 05 17:56:59.570504 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:59.570477    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:56:59.589613 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:56:59.589559    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" podStartSLOduration=1.589542553 podStartE2EDuration="1.589542553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:56:59.588328289 +0000 UTC m=+2678.121512528" watchObservedRunningTime="2026-08-05 17:56:59.589542553 +0000 UTC m=+2678.122726797"
Aug 05 17:57:01.997922 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:01.997890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:02.604913 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:02.604881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:03.333095 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:03.333061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:09.419745 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:09.419710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:20.770100 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:20.770063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:22.068370 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:22.068333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:22.078612 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:22.078582    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:57:22.082603 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:22.082582    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 17:57:22.083248 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:22.083225    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:57:22.087361 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:22.087344    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 17:57:25.007074 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:25.007038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:25.914461 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:25.914422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:31.581901 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:31.581865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:57:33.043566 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:33.043514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:40.527261 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:40.527217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:46.808660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:46.808630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:57:46.909471 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:57:46.909440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:05.990425 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:05.990359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:27.543619 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:27.543578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:27.639350 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:27.639294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:27.744414 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:27.744379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:27.839463 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:27.839387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:32.176392 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:32.176364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:32.616394 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:32.616356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:33.998959 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:33.998925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:45.090919 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.090843    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"]
Aug 05 17:58:45.094663 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.094643    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.098356 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.098326    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Aug 05 17:58:45.098476 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.098329    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Aug 05 17:58:45.098476 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.098393    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"default-dockercfg-8gnvz\""
Aug 05 17:58:45.098476 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.098333    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Aug 05 17:58:45.110460 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.110434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"]
Aug 05 17:58:45.128276 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.128253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:45.168451 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.168426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.168545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.168460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.168545 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.168485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.168640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.168574    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.168640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.168594    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.168640 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.168613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mrk\" (UniqueName: \"kubernetes.io/projected/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-kube-api-access-q6mrk\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269452 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269456    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269660 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mrk\" (UniqueName: \"kubernetes.io/projected/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-kube-api-access-q6mrk\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.269974 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269950    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.270044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.269993    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.270044 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.270012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.271698 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.271680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.271995 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.271978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.277227 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.277203    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mrk\" (UniqueName: \"kubernetes.io/projected/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-kube-api-access-q6mrk\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.405154 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.405089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:45.533847 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:45.533756    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"]
Aug 05 17:58:45.536539 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:58:45.536510    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef0d9f0_6d82_45ec_a4b1_9796891b28c9.slice/crio-77abe52c8a38d53bfa8b3a413b650f5654f38ae07faa74a4a1e0ed0d9cb29315 WatchSource:0}: Error finding container 77abe52c8a38d53bfa8b3a413b650f5654f38ae07faa74a4a1e0ed0d9cb29315: Status 404 returned error can't find the container with id 77abe52c8a38d53bfa8b3a413b650f5654f38ae07faa74a4a1e0ed0d9cb29315
Aug 05 17:58:46.063590 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:46.063555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" event={"ID":"aef0d9f0-6d82-45ec-a4b1-9796891b28c9","Type":"ContainerStarted","Data":"a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"}
Aug 05 17:58:46.063790 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:46.063623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" event={"ID":"aef0d9f0-6d82-45ec-a4b1-9796891b28c9","Type":"ContainerStarted","Data":"77abe52c8a38d53bfa8b3a413b650f5654f38ae07faa74a4a1e0ed0d9cb29315"}
Aug 05 17:58:46.081063 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:46.081013    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" podStartSLOduration=1.080998495 podStartE2EDuration="1.080998495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:58:46.079798477 +0000 UTC m=+2784.612982725" watchObservedRunningTime="2026-08-05 17:58:46.080998495 +0000 UTC m=+2784.614182731"
Aug 05 17:58:46.405308 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:46.405222    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:47.135362 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:47.135331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:47.140859 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:47.140832    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"]
Aug 05 17:58:47.205599 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:47.205567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:58:51.580800 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:51.580763    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:58:55.405537 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:55.405442    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:55.418203 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:55.418169    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:58:56.120886 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:58:56.120860    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 17:59:01.580763 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:01.580714    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 17:59:02.161110 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.161071    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerName="main" containerID="cri-o://aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af" gracePeriod=45
Aug 05 17:59:02.405136 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.405109    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:59:02.418284 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.418228    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nwwt\" (UniqueName: \"kubernetes.io/projected/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-kube-api-access-4nwwt\") pod \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") "
Aug 05 17:59:02.418391 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.418287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-tls-certs\") pod \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\" (UID: \"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba\") "
Aug 05 17:59:02.420485 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.420445    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" (UID: "94cfb912-8a4d-43f6-b1d8-961b3f02f1ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:59:02.420752 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.420710    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-kube-api-access-4nwwt" (OuterVolumeSpecName: "kube-api-access-4nwwt") pod "94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" (UID: "94cfb912-8a4d-43f6-b1d8-961b3f02f1ba"). InnerVolumeSpecName "kube-api-access-4nwwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:02.518931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.518906    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nwwt\" (UniqueName: \"kubernetes.io/projected/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-kube-api-access-4nwwt\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:59:02.518931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:02.518927    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:59:03.142589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.142553    2562 generic.go:356] "Generic (PLEG): container finished" podID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerID="aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af" exitCode=0
Aug 05 17:59:03.142589 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.142591    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"
Aug 05 17:59:03.143126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.142640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" event={"ID":"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba","Type":"ContainerDied","Data":"aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af"}
Aug 05 17:59:03.143126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.142696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf" event={"ID":"94cfb912-8a4d-43f6-b1d8-961b3f02f1ba","Type":"ContainerDied","Data":"c4b82181a04c586babb8eb07f3a772c88873b5b668bc439132c6b2b2a42b9dea"}
Aug 05 17:59:03.143126 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.142732    2562 scope.go:117] "RemoveContainer" containerID="aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af"
Aug 05 17:59:03.151986 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.151970    2562 scope.go:117] "RemoveContainer" containerID="aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af"
Aug 05 17:59:03.152245 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:03.152228    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af\": container with ID starting with aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af not found: ID does not exist" containerID="aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af"
Aug 05 17:59:03.152331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.152255    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af"} err="failed to get container status \"aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af\": rpc error: code = NotFound desc = could not find container \"aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af\": container with ID starting with aa7854e323eea8bf88819efe42c09afdbbb3ca4067838b027eecc080a13a53af not found: ID does not exist"
Aug 05 17:59:03.163932 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.163913    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"]
Aug 05 17:59:03.166971 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:03.166952    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schegh4lf"]
Aug 05 17:59:04.102835 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:04.102799    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" path="/var/lib/kubelet/pods/94cfb912-8a4d-43f6-b1d8-961b3f02f1ba/volumes"
Aug 05 17:59:05.125937 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:05.125907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:05.787595 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:05.787560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:05.882972 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:05.882937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:07.976030 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:07.975994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:08.076875 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:08.076825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:08.176524 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:08.176492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:09.490192 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:09.490159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:12.257939 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:12.257908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:13.788954 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:13.788921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:13.841961 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:13.841929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:15.480907 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:15.480876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:15.580737 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:15.580703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:18.294862 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:18.294831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:18.394759 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:18.394732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:21.621246 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.621209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:21.791552 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.791521    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"]
Aug 05 17:59:21.791997 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.791978    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerName="main"
Aug 05 17:59:21.792051 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.792000    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerName="main"
Aug 05 17:59:21.792094 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.792060    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="94cfb912-8a4d-43f6-b1d8-961b3f02f1ba" containerName="main"
Aug 05 17:59:21.796499 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.796479    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:21.799017 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.798997    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"e2e-del-blocked-svc-kserve-self-signed-certs\""
Aug 05 17:59:21.800504 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.800485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"default-dockercfg-jxr2m\""
Aug 05 17:59:21.800504 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.800496    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"e2e-del-blocked-svc-epp-sa-dockercfg-8njqp\""
Aug 05 17:59:21.800635 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.800501    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"openshift-service-ca.crt\""
Aug 05 17:59:21.800635 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.800517    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"kube-root-ca.crt\""
Aug 05 17:59:21.804893 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.804874    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"]
Aug 05 17:59:21.811166 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.811141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:21.865720 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.865683    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq9mr\" (UniqueName: \"kubernetes.io/projected/5e3f709e-3f42-4bd1-a751-3a5b194cf926-kube-api-access-pq9mr\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:21.865838 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.865731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e3f709e-3f42-4bd1-a751-3a5b194cf926-tls-certs\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:21.966732 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.966696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq9mr\" (UniqueName: \"kubernetes.io/projected/5e3f709e-3f42-4bd1-a751-3a5b194cf926-kube-api-access-pq9mr\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:21.966846 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.966742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e3f709e-3f42-4bd1-a751-3a5b194cf926-tls-certs\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:21.969525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.969506    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"e2e-del-blocked-svc-kserve-self-signed-certs\""
Aug 05 17:59:21.977009 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.976989    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"kube-root-ca.crt\""
Aug 05 17:59:21.979240 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.979220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e3f709e-3f42-4bd1-a751-3a5b194cf926-tls-certs\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:21.987083 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.987066    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"openshift-service-ca.crt\""
Aug 05 17:59:21.997931 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:21.997912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq9mr\" (UniqueName: \"kubernetes.io/projected/5e3f709e-3f42-4bd1-a751-3a5b194cf926-kube-api-access-pq9mr\") pod \"e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") " pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:22.110596 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:22.110573    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"default-dockercfg-jxr2m\""
Aug 05 17:59:22.121192 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:22.121173    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-config-deletion-blocked-when-referenced-0f029b85\"/\"e2e-del-blocked-svc-epp-sa-dockercfg-8njqp\""
Aug 05 17:59:22.129286 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:22.129270    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:22.266780 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:22.266737    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"]
Aug 05 17:59:22.272307 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:59:22.272280    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3f709e_3f42_4bd1_a751_3a5b194cf926.slice/crio-bb70ec65a7dde9b54a3f90c1334a6a2440b0e7d037bb7b6a7e37d78c45ca9fc6 WatchSource:0}: Error finding container bb70ec65a7dde9b54a3f90c1334a6a2440b0e7d037bb7b6a7e37d78c45ca9fc6: Status 404 returned error can't find the container with id bb70ec65a7dde9b54a3f90c1334a6a2440b0e7d037bb7b6a7e37d78c45ca9fc6
Aug 05 17:59:23.234281 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:23.234247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh" event={"ID":"5e3f709e-3f42-4bd1-a751-3a5b194cf926","Type":"ContainerStarted","Data":"89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab"}
Aug 05 17:59:23.234697 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:23.234295    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh" event={"ID":"5e3f709e-3f42-4bd1-a751-3a5b194cf926","Type":"ContainerStarted","Data":"bb70ec65a7dde9b54a3f90c1334a6a2440b0e7d037bb7b6a7e37d78c45ca9fc6"}
Aug 05 17:59:23.234697 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:23.234372    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:23.251983 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:23.251936    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh" podStartSLOduration=2.251921558 podStartE2EDuration="2.251921558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:59:23.251176286 +0000 UTC m=+2821.784360534" watchObservedRunningTime="2026-08-05 17:59:23.251921558 +0000 UTC m=+2821.785105793"
Aug 05 17:59:24.430894 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:24.430857    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"]
Aug 05 17:59:24.437916 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:24.437889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:24.487578 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:24.487548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:26.494331 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:26.494294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:26.597428 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:26.597395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:26.695075 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:26.695044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:30.743162 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:30.743128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:30.845629 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:30.845596    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:30.944089 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:30.944060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:31.562771 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:31.562735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:35.082225 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:35.082188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:35.250587 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:35.250532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:35.354581 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:35.354510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:36.693002 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:36.692971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:37.034435 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:37.034402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:37.134696 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:37.134665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:40.261693 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.261635    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh" podUID="5e3f709e-3f42-4bd1-a751-3a5b194cf926" containerName="main" containerID="cri-o://89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab" gracePeriod=45
Aug 05 17:59:40.516991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.516934    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:40.617187 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.617154    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq9mr\" (UniqueName: \"kubernetes.io/projected/5e3f709e-3f42-4bd1-a751-3a5b194cf926-kube-api-access-pq9mr\") pod \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") "
Aug 05 17:59:40.617352 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.617200    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e3f709e-3f42-4bd1-a751-3a5b194cf926-tls-certs\") pod \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\" (UID: \"5e3f709e-3f42-4bd1-a751-3a5b194cf926\") "
Aug 05 17:59:40.619419 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.619391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e3f709e-3f42-4bd1-a751-3a5b194cf926-kube-api-access-pq9mr" (OuterVolumeSpecName: "kube-api-access-pq9mr") pod "5e3f709e-3f42-4bd1-a751-3a5b194cf926" (UID: "5e3f709e-3f42-4bd1-a751-3a5b194cf926"). InnerVolumeSpecName "kube-api-access-pq9mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:40.619525 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.619397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e3f709e-3f42-4bd1-a751-3a5b194cf926-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e3f709e-3f42-4bd1-a751-3a5b194cf926" (UID: "5e3f709e-3f42-4bd1-a751-3a5b194cf926"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:59:40.718069 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.718042    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq9mr\" (UniqueName: \"kubernetes.io/projected/5e3f709e-3f42-4bd1-a751-3a5b194cf926-kube-api-access-pq9mr\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:59:40.718069 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.718068    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e3f709e-3f42-4bd1-a751-3a5b194cf926-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 17:59:40.916424 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.916351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:40.981974 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:40.981943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:41.316417 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.316386    2562 generic.go:356] "Generic (PLEG): container finished" podID="5e3f709e-3f42-4bd1-a751-3a5b194cf926" containerID="89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab" exitCode=0
Aug 05 17:59:41.316843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.316430    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"
Aug 05 17:59:41.316843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.316476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh" event={"ID":"5e3f709e-3f42-4bd1-a751-3a5b194cf926","Type":"ContainerDied","Data":"89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab"}
Aug 05 17:59:41.316843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.316526    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh" event={"ID":"5e3f709e-3f42-4bd1-a751-3a5b194cf926","Type":"ContainerDied","Data":"bb70ec65a7dde9b54a3f90c1334a6a2440b0e7d037bb7b6a7e37d78c45ca9fc6"}
Aug 05 17:59:41.316843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.316549    2562 scope.go:117] "RemoveContainer" containerID="89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab"
Aug 05 17:59:41.325744 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.325728    2562 scope.go:117] "RemoveContainer" containerID="89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab"
Aug 05 17:59:41.325983 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:41.325959    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab\": container with ID starting with 89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab not found: ID does not exist" containerID="89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab"
Aug 05 17:59:41.326083 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.325989    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab"} err="failed to get container status \"89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab\": rpc error: code = NotFound desc = could not find container \"89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab\": container with ID starting with 89b4997e78180540277bd1f1f8bc7771c8b70b202994781044e4bdef24a7b6ab not found: ID does not exist"
Aug 05 17:59:41.337361 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.337340    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"]
Aug 05 17:59:41.342200 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:41.342180    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-config-deletion-blocked-when-referenced-0f029b85/e2e-del-blocked-svc-kserve-router-scheduler-5c74dd894f-4lrkh"]
Aug 05 17:59:42.102726 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:42.102693    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e3f709e-3f42-4bd1-a751-3a5b194cf926" path="/var/lib/kubelet/pods/5e3f709e-3f42-4bd1-a751-3a5b194cf926/volumes"
Aug 05 17:59:45.755843 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.755807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:45.919592 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.919561    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"]
Aug 05 17:59:45.919947 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.919933    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e3f709e-3f42-4bd1-a751-3a5b194cf926" containerName="main"
Aug 05 17:59:45.919991 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.919948    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e3f709e-3f42-4bd1-a751-3a5b194cf926" containerName="main"
Aug 05 17:59:45.920076 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.920069    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e3f709e-3f42-4bd1-a751-3a5b194cf926" containerName="main"
Aug 05 17:59:45.924383 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.924367    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:45.928000 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.927975    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b\"/\"kube-root-ca.crt\""
Aug 05 17:59:45.928125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.927975    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b\"/\"default-dockercfg-fqd9v\""
Aug 05 17:59:45.928125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.928030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b\"/\"e2e-wk-cfg-implicit-svc-kserve-self-signed-certs\""
Aug 05 17:59:45.928125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.928078    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b\"/\"e2e-wk-cfg-implicit-svc-epp-sa-dockercfg-wlpkz\""
Aug 05 17:59:45.928125 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.927977    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b\"/\"openshift-service-ca.crt\""
Aug 05 17:59:45.934196 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.934173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"]
Aug 05 17:59:45.967795 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:45.967763    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:46.060576 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.060501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs\") pod \"e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") " pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.060713 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.060575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nr6f\" (UniqueName: \"kubernetes.io/projected/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-kube-api-access-6nr6f\") pod \"e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") " pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.161891 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.161858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nr6f\" (UniqueName: \"kubernetes.io/projected/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-kube-api-access-6nr6f\") pod \"e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") " pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.162031 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.161978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs\") pod \"e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") " pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.164536 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.164515    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs\") pod \"e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") " pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.169527 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.169500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nr6f\" (UniqueName: \"kubernetes.io/projected/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-kube-api-access-6nr6f\") pod \"e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") " pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.237015 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.236989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:46.381771 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:46.381740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"]
Aug 05 17:59:46.384731 ip-10-0-131-218 kubenswrapper[2562]: W0805 17:59:46.384705    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde6e6c7_7ba5_46e9_b3a8_620bdfa78eba.slice/crio-00b26a90557585885d4de4289f3e0b06b02926a1e618c81b368043f4fac52495 WatchSource:0}: Error finding container 00b26a90557585885d4de4289f3e0b06b02926a1e618c81b368043f4fac52495: Status 404 returned error can't find the container with id 00b26a90557585885d4de4289f3e0b06b02926a1e618c81b368043f4fac52495
Aug 05 17:59:47.347074 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:47.347040    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" event={"ID":"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba","Type":"ContainerStarted","Data":"baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446"}
Aug 05 17:59:47.347470 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:47.347097    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" event={"ID":"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba","Type":"ContainerStarted","Data":"00b26a90557585885d4de4289f3e0b06b02926a1e618c81b368043f4fac52495"}
Aug 05 17:59:47.347470 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:47.347125    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 17:59:47.363351 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:47.363287    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" podStartSLOduration=2.363272717 podStartE2EDuration="2.363272717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:59:47.36255806 +0000 UTC m=+2845.895742317" watchObservedRunningTime="2026-08-05 17:59:47.363272717 +0000 UTC m=+2845.896456962"
Aug 05 17:59:48.171626 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:48.171584    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"]
Aug 05 17:59:48.175638 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:48.175604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:48.228725 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:48.228696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:48.351521 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:48.351492    2562 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" secret="" err="secret \"e2e-wk-cfg-implicit-svc-epp-sa-dockercfg-wlpkz\" not found"
Aug 05 17:59:48.482594 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:48.482566    2562 secret.go:189] Couldn't get secret e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-self-signed-certs: secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:48.482751 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:48.482633    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs podName:dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba nodeName:}" failed. No retries permitted until 2026-08-05 17:59:48.982616095 +0000 UTC m=+2847.515800317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs") pod "e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba") : secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:48.986741 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:48.986712    2562 secret.go:189] Couldn't get secret e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-self-signed-certs: secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:48.986902 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:48.986773    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs podName:dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba nodeName:}" failed. No retries permitted until 2026-08-05 17:59:49.986760251 +0000 UTC m=+2848.519944475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs") pod "e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba") : secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:49.642185 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:49.642152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:49.742752 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:49.742724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:49.848539 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:49.848512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:49.994964 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:49.994934    2562 secret.go:189] Couldn't get secret e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-self-signed-certs: secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:49.995121 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:49.994994    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs podName:dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba nodeName:}" failed. No retries permitted until 2026-08-05 17:59:51.994981465 +0000 UTC m=+2850.528165688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs") pod "e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba") : secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:52.012743 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:52.012709    2562 secret.go:189] Couldn't get secret e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-self-signed-certs: secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:52.013123 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:52.012785    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs podName:dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba nodeName:}" failed. No retries permitted until 2026-08-05 17:59:56.012771445 +0000 UTC m=+2854.545955672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs") pod "e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba") : secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:54.720905 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:54.720868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:56.047863 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:56.047830    2562 secret.go:189] Couldn't get secret e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-self-signed-certs: secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:56.048234 ip-10-0-131-218 kubenswrapper[2562]: E0805 17:59:56.047900    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs podName:dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba nodeName:}" failed. No retries permitted until 2026-08-05 18:00:04.047887596 +0000 UTC m=+2862.581071820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs") pod "e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba") : secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 17:59:57.703121 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:57.703088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 17:59:57.886858 ip-10-0-131-218 kubenswrapper[2562]: I0805 17:59:57.886823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:00:03.792040 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:03.792004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:00:04.119490 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:00:04.119419    2562 secret.go:189] Couldn't get secret e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-self-signed-certs: secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 18:00:04.119643 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:00:04.119494    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs podName:dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba nodeName:}" failed. No retries permitted until 2026-08-05 18:00:20.11948009 +0000 UTC m=+2878.652664313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs") pod "e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba") : secret "e2e-wk-cfg-implicit-svc-kserve-self-signed-certs" not found
Aug 05 18:00:04.373849 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.373765    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" podUID="dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" containerName="main" containerID="cri-o://baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446" gracePeriod=45
Aug 05 18:00:04.617777 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.617746    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 18:00:04.723847 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.723805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs\") pod \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") "
Aug 05 18:00:04.724009 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.723949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nr6f\" (UniqueName: \"kubernetes.io/projected/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-kube-api-access-6nr6f\") pod \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\" (UID: \"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba\") "
Aug 05 18:00:04.726008 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.725984    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:00:04.726080 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.726027    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-kube-api-access-6nr6f" (OuterVolumeSpecName: "kube-api-access-6nr6f") pod "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" (UID: "dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba"). InnerVolumeSpecName "kube-api-access-6nr6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:00:04.824877 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.824851    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:00:04.824877 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:04.824875    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nr6f\" (UniqueName: \"kubernetes.io/projected/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba-kube-api-access-6nr6f\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:00:05.431027 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.430991    2562 generic.go:356] "Generic (PLEG): container finished" podID="dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" containerID="baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446" exitCode=0
Aug 05 18:00:05.431183 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.431031    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"
Aug 05 18:00:05.431183 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.431075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" event={"ID":"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba","Type":"ContainerDied","Data":"baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446"}
Aug 05 18:00:05.431183 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.431127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk" event={"ID":"dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba","Type":"ContainerDied","Data":"00b26a90557585885d4de4289f3e0b06b02926a1e618c81b368043f4fac52495"}
Aug 05 18:00:05.431183 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.431154    2562 scope.go:117] "RemoveContainer" containerID="baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446"
Aug 05 18:00:05.439713 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.439693    2562 scope.go:117] "RemoveContainer" containerID="baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446"
Aug 05 18:00:05.439969 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:00:05.439950    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446\": container with ID starting with baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446 not found: ID does not exist" containerID="baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446"
Aug 05 18:00:05.440062 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.439974    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446"} err="failed to get container status \"baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446\": rpc error: code = NotFound desc = could not find container \"baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446\": container with ID starting with baf6ce136734ebcaaeffcec0df66e185e277174b17156b5fad0c2e8e0ddbc446 not found: ID does not exist"
Aug 05 18:00:05.453338 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.453302    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"]
Aug 05 18:00:05.456785 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:05.456762    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-well-known-config-deletion-blocked-by-implici-6b767e7b/e2e-wk-cfg-implicit-svc-kserve-router-scheduler-6d99c88894fxnzk"]
Aug 05 18:00:06.101635 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:06.101608    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" path="/var/lib/kubelet/pods/dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba/volumes"
Aug 05 18:00:08.805089 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:08.805056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:00:31.804845 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:00:31.804805    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:06.828212 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:06.828175    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"]
Aug 05 18:01:06.837145 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:01:06.837111    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 18:01:06.837145 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:01:06.837145 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:01:06.837145 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54" cmd=["/bin/sleep","15"]
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:06.837160    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" message="" err=<
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:01:06.837172    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" podUID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" containerName="main" containerID="cri-o://a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"
Aug 05 18:01:06.837406 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:06.837218    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" podUID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" containerName="main" containerID="cri-o://a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54" gracePeriod=60
Aug 05 18:01:06.850373 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:06.850351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:07.076150 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.076124    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 18:01:07.223608 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-model-cache\") pod \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") "
Aug 05 18:01:07.223799 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223661    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tmp-dir\") pod \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") "
Aug 05 18:01:07.223799 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223686    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tls-certs\") pod \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") "
Aug 05 18:01:07.223799 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223720    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6mrk\" (UniqueName: \"kubernetes.io/projected/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-kube-api-access-q6mrk\") pod \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") "
Aug 05 18:01:07.223799 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223768    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-home\") pod \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") "
Aug 05 18:01:07.223799 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223790    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-dshm\") pod \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\" (UID: \"aef0d9f0-6d82-45ec-a4b1-9796891b28c9\") "
Aug 05 18:01:07.224063 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223886    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "aef0d9f0-6d82-45ec-a4b1-9796891b28c9" (UID: "aef0d9f0-6d82-45ec-a4b1-9796891b28c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:01:07.224063 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.223933    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aef0d9f0-6d82-45ec-a4b1-9796891b28c9" (UID: "aef0d9f0-6d82-45ec-a4b1-9796891b28c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:01:07.224063 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.224051    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-home" (OuterVolumeSpecName: "home") pod "aef0d9f0-6d82-45ec-a4b1-9796891b28c9" (UID: "aef0d9f0-6d82-45ec-a4b1-9796891b28c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:01:07.224245 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.224065    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:01:07.224245 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.224079    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:01:07.225849 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.225821    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-dshm" (OuterVolumeSpecName: "dshm") pod "aef0d9f0-6d82-45ec-a4b1-9796891b28c9" (UID: "aef0d9f0-6d82-45ec-a4b1-9796891b28c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:01:07.225934 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.225846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aef0d9f0-6d82-45ec-a4b1-9796891b28c9" (UID: "aef0d9f0-6d82-45ec-a4b1-9796891b28c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:01:07.225934 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.225870    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-kube-api-access-q6mrk" (OuterVolumeSpecName: "kube-api-access-q6mrk") pod "aef0d9f0-6d82-45ec-a4b1-9796891b28c9" (UID: "aef0d9f0-6d82-45ec-a4b1-9796891b28c9"). InnerVolumeSpecName "kube-api-access-q6mrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:01:07.324609 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.324576    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6mrk\" (UniqueName: \"kubernetes.io/projected/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-kube-api-access-q6mrk\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:01:07.324609 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.324605    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:01:07.324609 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.324615    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:01:07.324805 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.324624    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef0d9f0-6d82-45ec-a4b1-9796891b28c9-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:01:07.716875 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.716839    2562 generic.go:356] "Generic (PLEG): container finished" podID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" containerID="a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54" exitCode=0
Aug 05 18:01:07.717059 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.716890    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"
Aug 05 18:01:07.717059 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.716930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" event={"ID":"aef0d9f0-6d82-45ec-a4b1-9796891b28c9","Type":"ContainerDied","Data":"a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"}
Aug 05 18:01:07.717059 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.716999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2" event={"ID":"aef0d9f0-6d82-45ec-a4b1-9796891b28c9","Type":"ContainerDied","Data":"77abe52c8a38d53bfa8b3a413b650f5654f38ae07faa74a4a1e0ed0d9cb29315"}
Aug 05 18:01:07.717059 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.717031    2562 scope.go:117] "RemoveContainer" containerID="a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"
Aug 05 18:01:07.725915 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.725898    2562 scope.go:117] "RemoveContainer" containerID="a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"
Aug 05 18:01:07.726157 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:01:07.726139    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54\": container with ID starting with a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54 not found: ID does not exist" containerID="a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"
Aug 05 18:01:07.726201 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.726162    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54"} err="failed to get container status \"a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54\": rpc error: code = NotFound desc = could not find container \"a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54\": container with ID starting with a1bd052f10a3efec12c5bb2151090b6ed42b210d8b5922bc17f0a59ccdbb8f54 not found: ID does not exist"
Aug 05 18:01:07.738013 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.737994    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"]
Aug 05 18:01:07.743627 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:07.743599    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5d6ff58997z28h2"]
Aug 05 18:01:08.102725 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:08.102648    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" path="/var/lib/kubelet/pods/aef0d9f0-6d82-45ec-a4b1-9796891b28c9/volumes"
Aug 05 18:01:11.901981 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:11.901949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:11.995476 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:11.995434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:16.883072 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:16.883020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:20.641395 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.641358    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:20.908084 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908004    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"]
Aug 05 18:01:20.908583 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908562    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" containerName="main"
Aug 05 18:01:20.908635 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908588    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" containerName="main"
Aug 05 18:01:20.908672 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908638    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" containerName="main"
Aug 05 18:01:20.908672 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908647    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" containerName="main"
Aug 05 18:01:20.908734 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908726    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde6e6c7-7ba5-46e9-b3a8-620bdfa78eba" containerName="main"
Aug 05 18:01:20.908768 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.908743    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aef0d9f0-6d82-45ec-a4b1-9796891b28c9" containerName="main"
Aug 05 18:01:20.913072 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.913048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:20.916935 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.916908    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Aug 05 18:01:20.917057 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.916963    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-b9cnl\""
Aug 05 18:01:20.917057 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.916910    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Aug 05 18:01:20.917169 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.916913    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-64xwq\""
Aug 05 18:01:20.917169 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.916909    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Aug 05 18:01:20.919813 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.919787    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"]
Aug 05 18:01:20.940686 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.940657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ktj\" (UniqueName: \"kubernetes.io/projected/06c41b12-30a3-47b1-9397-88c0567b0c60-kube-api-access-l5ktj\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:20.940778 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.940702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c41b12-30a3-47b1-9397-88c0567b0c60-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:20.945902 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:20.945871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:21.041961 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.041929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ktj\" (UniqueName: \"kubernetes.io/projected/06c41b12-30a3-47b1-9397-88c0567b0c60-kube-api-access-l5ktj\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:21.042111 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.041977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c41b12-30a3-47b1-9397-88c0567b0c60-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:21.044414 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.044391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c41b12-30a3-47b1-9397-88c0567b0c60-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:21.050259 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.050229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ktj\" (UniqueName: \"kubernetes.io/projected/06c41b12-30a3-47b1-9397-88c0567b0c60-kube-api-access-l5ktj\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:21.225489 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.225455    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:21.353512 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.353481    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"]
Aug 05 18:01:21.357475 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:01:21.357442    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c41b12_30a3_47b1_9397_88c0567b0c60.slice/crio-ca206bea10096ca81ceac8d921147910bed089e2898fefaee2fe8b7ff696ba18 WatchSource:0}: Error finding container ca206bea10096ca81ceac8d921147910bed089e2898fefaee2fe8b7ff696ba18: Status 404 returned error can't find the container with id ca206bea10096ca81ceac8d921147910bed089e2898fefaee2fe8b7ff696ba18
Aug 05 18:01:21.783758 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.783715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" event={"ID":"06c41b12-30a3-47b1-9397-88c0567b0c60","Type":"ContainerStarted","Data":"46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c"}
Aug 05 18:01:21.784222 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.783786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" event={"ID":"06c41b12-30a3-47b1-9397-88c0567b0c60","Type":"ContainerStarted","Data":"ca206bea10096ca81ceac8d921147910bed089e2898fefaee2fe8b7ff696ba18"}
Aug 05 18:01:21.784222 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.783913    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:21.801605 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:21.801559    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" podStartSLOduration=1.801543278 podStartE2EDuration="1.801543278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:21.79941275 +0000 UTC m=+2940.332597010" watchObservedRunningTime="2026-08-05 18:01:21.801543278 +0000 UTC m=+2940.334727523"
Aug 05 18:01:31.507076 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:31.507034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:32.286280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:32.286248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:38.307205 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:38.307173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:39.135245 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:39.135215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:44.835058 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:44.834973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:52.790132 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:52.790103    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:01:54.794886 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:54.794855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:01:59.409353 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:01:59.409307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:00.191223 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:00.191192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:00.247191 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:00.247112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:22.108153 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:22.108122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:02:22.113078 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:22.113051    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:02:22.113291 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:22.113271    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:02:22.119674 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:22.119655    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:02:25.450415 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.450373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:25.585550 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.585515    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"]
Aug 05 18:02:25.588770 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.588751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.592412 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.592341    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Aug 05 18:02:25.592412 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.592347    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Aug 05 18:02:25.592412 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.592385    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-576ls\""
Aug 05 18:02:25.592709 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.592421    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Aug 05 18:02:25.592709 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.592460    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-jbxpc\""
Aug 05 18:02:25.597430 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.597408    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"]
Aug 05 18:02:25.663506 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.663471    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.663650 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.663524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbc7w\" (UniqueName: \"kubernetes.io/projected/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-kube-api-access-bbc7w\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.764420 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.764351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.764420 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.764395    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbc7w\" (UniqueName: \"kubernetes.io/projected/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-kube-api-access-bbc7w\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.766783 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.766760    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.772173 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.772150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbc7w\" (UniqueName: \"kubernetes.io/projected/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-kube-api-access-bbc7w\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:25.900432 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:25.900402    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:26.029409 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:26.029384    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"]
Aug 05 18:02:26.031692 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:02:26.031665    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f0de14_25a6_41cc_a8f8_12b3c7376c0e.slice/crio-d666517bb4b1880101d8d76e2995e44f30b18e41f16e1b2df9c9b10be71582c1 WatchSource:0}: Error finding container d666517bb4b1880101d8d76e2995e44f30b18e41f16e1b2df9c9b10be71582c1: Status 404 returned error can't find the container with id d666517bb4b1880101d8d76e2995e44f30b18e41f16e1b2df9c9b10be71582c1
Aug 05 18:02:26.082555 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:26.082520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" event={"ID":"18f0de14-25a6-41cc-a8f8-12b3c7376c0e","Type":"ContainerStarted","Data":"d666517bb4b1880101d8d76e2995e44f30b18e41f16e1b2df9c9b10be71582c1"}
Aug 05 18:02:27.088850 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:27.088818    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" event={"ID":"18f0de14-25a6-41cc-a8f8-12b3c7376c0e","Type":"ContainerStarted","Data":"07ca088e761f03a40a4a72c1e6f6b75b1d13f93b04da3c5cc84859666d5b6927"}
Aug 05 18:02:27.089255 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:27.088935    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:02:27.104330 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:27.104280    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" podStartSLOduration=2.10426649 podStartE2EDuration="2.10426649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:02:27.104106208 +0000 UTC m=+3005.637290454" watchObservedRunningTime="2026-08-05 18:02:27.10426649 +0000 UTC m=+3005.637450735"
Aug 05 18:02:27.345607 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:27.345528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:45.812443 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:45.812400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:02:58.095654 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:02:58.095622    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:03:11.952506 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:11.952471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:25.314111 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:25.314078    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"]
Aug 05 18:03:25.314601 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:25.314571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:25.410360 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:25.410305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:32.790280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:32.790237    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" podUID="06c41b12-30a3-47b1-9397-88c0567b0c60" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:03:40.333470 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.333404    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" podUID="06c41b12-30a3-47b1-9397-88c0567b0c60" containerName="main" containerID="cri-o://46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c" gracePeriod=45
Aug 05 18:03:40.579061 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.579035    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:03:40.662108 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.661993    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c41b12-30a3-47b1-9397-88c0567b0c60-tls-certs\") pod \"06c41b12-30a3-47b1-9397-88c0567b0c60\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") "
Aug 05 18:03:40.662108 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.662095    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5ktj\" (UniqueName: \"kubernetes.io/projected/06c41b12-30a3-47b1-9397-88c0567b0c60-kube-api-access-l5ktj\") pod \"06c41b12-30a3-47b1-9397-88c0567b0c60\" (UID: \"06c41b12-30a3-47b1-9397-88c0567b0c60\") "
Aug 05 18:03:40.664482 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.664451    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06c41b12-30a3-47b1-9397-88c0567b0c60-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06c41b12-30a3-47b1-9397-88c0567b0c60" (UID: "06c41b12-30a3-47b1-9397-88c0567b0c60"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:03:40.664613 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.664589    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c41b12-30a3-47b1-9397-88c0567b0c60-kube-api-access-l5ktj" (OuterVolumeSpecName: "kube-api-access-l5ktj") pod "06c41b12-30a3-47b1-9397-88c0567b0c60" (UID: "06c41b12-30a3-47b1-9397-88c0567b0c60"). InnerVolumeSpecName "kube-api-access-l5ktj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:03:40.762869 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.762835    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06c41b12-30a3-47b1-9397-88c0567b0c60-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:03:40.762869 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:40.762863    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5ktj\" (UniqueName: \"kubernetes.io/projected/06c41b12-30a3-47b1-9397-88c0567b0c60-kube-api-access-l5ktj\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.436089 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.436050    2562 generic.go:356] "Generic (PLEG): container finished" podID="06c41b12-30a3-47b1-9397-88c0567b0c60" containerID="46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c" exitCode=0
Aug 05 18:03:41.436089 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.436089    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"
Aug 05 18:03:41.436577 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.436101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" event={"ID":"06c41b12-30a3-47b1-9397-88c0567b0c60","Type":"ContainerDied","Data":"46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c"}
Aug 05 18:03:41.436577 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.436151    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7" event={"ID":"06c41b12-30a3-47b1-9397-88c0567b0c60","Type":"ContainerDied","Data":"ca206bea10096ca81ceac8d921147910bed089e2898fefaee2fe8b7ff696ba18"}
Aug 05 18:03:41.436577 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.436173    2562 scope.go:117] "RemoveContainer" containerID="46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c"
Aug 05 18:03:41.445358 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.445336    2562 scope.go:117] "RemoveContainer" containerID="46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c"
Aug 05 18:03:41.445603 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:03:41.445582    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c\": container with ID starting with 46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c not found: ID does not exist" containerID="46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c"
Aug 05 18:03:41.445668 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.445611    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c"} err="failed to get container status \"46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c\": rpc error: code = NotFound desc = could not find container \"46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c\": container with ID starting with 46d9a0ab9c0167bdbb9ad8fc4ac83a36af9cceb0927ed7108d125954c4d9805c not found: ID does not exist"
Aug 05 18:03:41.458486 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.458463    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"]
Aug 05 18:03:41.463455 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:41.463431    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sched98x7"]
Aug 05 18:03:42.102325 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:42.102282    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06c41b12-30a3-47b1-9397-88c0567b0c60" path="/var/lib/kubelet/pods/06c41b12-30a3-47b1-9397-88c0567b0c60/volumes"
Aug 05 18:03:45.343496 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:45.343467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:45.439409 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:45.439363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:51.419464 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:51.419421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:53.233692 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.233658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:53.454361 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.454323    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"]
Aug 05 18:03:53.454678 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.454666    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c41b12-30a3-47b1-9397-88c0567b0c60" containerName="main"
Aug 05 18:03:53.454725 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.454681    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c41b12-30a3-47b1-9397-88c0567b0c60" containerName="main"
Aug 05 18:03:53.454770 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.454760    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c41b12-30a3-47b1-9397-88c0567b0c60" containerName="main"
Aug 05 18:03:53.458829 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.458805    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.462242 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.462215    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-wv7sp\""
Aug 05 18:03:53.462378 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.462286    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Aug 05 18:03:53.462378 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.462292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Aug 05 18:03:53.462554 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.462536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-ftz95\""
Aug 05 18:03:53.462724 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.462705    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Aug 05 18:03:53.467557 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.467534    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"]
Aug 05 18:03:53.477367 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.477333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:53.565852 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.565766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.565852 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.565835    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5pjn\" (UniqueName: \"kubernetes.io/projected/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-kube-api-access-z5pjn\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.667199 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.667161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5pjn\" (UniqueName: \"kubernetes.io/projected/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-kube-api-access-z5pjn\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.667387 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.667224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.669660 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.669631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.678116 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.678088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5pjn\" (UniqueName: \"kubernetes.io/projected/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-kube-api-access-z5pjn\") pod \"custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.770207 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.770174    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:53.906358 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:53.906309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"]
Aug 05 18:03:53.908512 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:03:53.908473    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff59d1fe_a01e_4f71_9a8e_36c675bd6fcf.slice/crio-b25e0c19985882cf3bf8e711fe6d1e9f9f009b2a18371333bf0937aa7d54d74a WatchSource:0}: Error finding container b25e0c19985882cf3bf8e711fe6d1e9f9f009b2a18371333bf0937aa7d54d74a: Status 404 returned error can't find the container with id b25e0c19985882cf3bf8e711fe6d1e9f9f009b2a18371333bf0937aa7d54d74a
Aug 05 18:03:54.500260 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:54.500222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" event={"ID":"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf","Type":"ContainerStarted","Data":"4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef"}
Aug 05 18:03:54.500694 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:54.500280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" event={"ID":"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf","Type":"ContainerStarted","Data":"b25e0c19985882cf3bf8e711fe6d1e9f9f009b2a18371333bf0937aa7d54d74a"}
Aug 05 18:03:54.500694 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:54.500352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:03:54.531274 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:54.531220    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" podStartSLOduration=1.531205608 podStartE2EDuration="1.531205608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:03:54.530083736 +0000 UTC m=+3093.063267981" watchObservedRunningTime="2026-08-05 18:03:54.531205608 +0000 UTC m=+3093.064389852"
Aug 05 18:03:57.062274 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:57.062241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:57.162298 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:57.162267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:03:59.280823 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:03:59.280787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:21.063927 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:21.063893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:21.162195 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:21.162158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:25.507148 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:25.507119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:04:27.462127 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:27.462093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:27.468866 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:27.468830    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"]
Aug 05 18:04:27.555367 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:27.555333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:28.094959 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:28.094918    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:04:38.095422 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:38.095376    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:04:42.498470 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.498428    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerName="main" containerID="cri-o://07ca088e761f03a40a4a72c1e6f6b75b1d13f93b04da3c5cc84859666d5b6927" gracePeriod=45
Aug 05 18:04:42.726401 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.726369    2562 generic.go:356] "Generic (PLEG): container finished" podID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerID="07ca088e761f03a40a4a72c1e6f6b75b1d13f93b04da3c5cc84859666d5b6927" exitCode=0
Aug 05 18:04:42.726576 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.726446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" event={"ID":"18f0de14-25a6-41cc-a8f8-12b3c7376c0e","Type":"ContainerDied","Data":"07ca088e761f03a40a4a72c1e6f6b75b1d13f93b04da3c5cc84859666d5b6927"}
Aug 05 18:04:42.746772 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.746750    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:04:42.759901 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.759850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-tls-certs\") pod \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") "
Aug 05 18:04:42.759901 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.759878    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbc7w\" (UniqueName: \"kubernetes.io/projected/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-kube-api-access-bbc7w\") pod \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\" (UID: \"18f0de14-25a6-41cc-a8f8-12b3c7376c0e\") "
Aug 05 18:04:42.762118 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.762096    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18f0de14-25a6-41cc-a8f8-12b3c7376c0e" (UID: "18f0de14-25a6-41cc-a8f8-12b3c7376c0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:04:42.762193 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.762111    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-kube-api-access-bbc7w" (OuterVolumeSpecName: "kube-api-access-bbc7w") pod "18f0de14-25a6-41cc-a8f8-12b3c7376c0e" (UID: "18f0de14-25a6-41cc-a8f8-12b3c7376c0e"). InnerVolumeSpecName "kube-api-access-bbc7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:42.860737 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.860708    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:04:42.860737 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:42.860736    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbc7w\" (UniqueName: \"kubernetes.io/projected/18f0de14-25a6-41cc-a8f8-12b3c7376c0e-kube-api-access-bbc7w\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:04:43.732663 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:43.732629    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"
Aug 05 18:04:43.732663 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:43.732638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf" event={"ID":"18f0de14-25a6-41cc-a8f8-12b3c7376c0e","Type":"ContainerDied","Data":"d666517bb4b1880101d8d76e2995e44f30b18e41f16e1b2df9c9b10be71582c1"}
Aug 05 18:04:43.733189 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:43.732700    2562 scope.go:117] "RemoveContainer" containerID="07ca088e761f03a40a4a72c1e6f6b75b1d13f93b04da3c5cc84859666d5b6927"
Aug 05 18:04:43.756194 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:43.756166    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"]
Aug 05 18:04:43.760489 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:43.760464    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-9xjkf"]
Aug 05 18:04:44.108555 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:44.108476    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" path="/var/lib/kubelet/pods/18f0de14-25a6-41cc-a8f8-12b3c7376c0e/volumes"
Aug 05 18:04:47.284091 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:47.284056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:47.383365 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:47.383335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:47.483809 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:47.483768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:50.183481 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:50.183437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:53.402417 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:53.402378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:54.898597 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:54.898563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:58.985672 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:58.985636    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:04:59.044580 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:04:59.044551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:58.447537 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:05:58.447504    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:05:58.459098 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:05:58.459062    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"]
Aug 05 18:05:58.503303 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:05:58.503276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:05.507436 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:05.507395    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" podUID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:06:13.478887 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.478832    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" podUID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" containerName="main" containerID="cri-o://4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef" gracePeriod=45
Aug 05 18:06:13.725357 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.725329    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:06:13.892200 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.892121    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5pjn\" (UniqueName: \"kubernetes.io/projected/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-kube-api-access-z5pjn\") pod \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") "
Aug 05 18:06:13.892200 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.892175    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-tls-certs\") pod \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\" (UID: \"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf\") "
Aug 05 18:06:13.894277 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.894254    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" (UID: "ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:06:13.894369 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.894271    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-kube-api-access-z5pjn" (OuterVolumeSpecName: "kube-api-access-z5pjn") pod "ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" (UID: "ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf"). InnerVolumeSpecName "kube-api-access-z5pjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:06:13.993297 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.993269    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:06:13.993297 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:13.993295    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5pjn\" (UniqueName: \"kubernetes.io/projected/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf-kube-api-access-z5pjn\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:06:14.148430 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.148358    2562 generic.go:356] "Generic (PLEG): container finished" podID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" containerID="4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef" exitCode=0
Aug 05 18:06:14.148430 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.148394    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"
Aug 05 18:06:14.148578 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.148445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" event={"ID":"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf","Type":"ContainerDied","Data":"4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef"}
Aug 05 18:06:14.148578 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.148495    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s" event={"ID":"ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf","Type":"ContainerDied","Data":"b25e0c19985882cf3bf8e711fe6d1e9f9f009b2a18371333bf0937aa7d54d74a"}
Aug 05 18:06:14.148578 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.148516    2562 scope.go:117] "RemoveContainer" containerID="4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef"
Aug 05 18:06:14.157210 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.157189    2562 scope.go:117] "RemoveContainer" containerID="4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef"
Aug 05 18:06:14.157477 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:06:14.157454    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef\": container with ID starting with 4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef not found: ID does not exist" containerID="4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef"
Aug 05 18:06:14.157565 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.157483    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef"} err="failed to get container status \"4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef\": rpc error: code = NotFound desc = could not find container \"4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef\": container with ID starting with 4bbe21c97727eb1ab90a7b0f7b8af5b5efc475a3fb7ced8a7a17a452db5fa7ef not found: ID does not exist"
Aug 05 18:06:14.165462 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.165443    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"]
Aug 05 18:06:14.174069 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:14.174049    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-c9cb668997g6s"]
Aug 05 18:06:16.101972 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:16.101940    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" path="/var/lib/kubelet/pods/ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf/volumes"
Aug 05 18:06:18.507095 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:18.507066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:18.600954 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:18.600920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:18.751226 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:18.751196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:18.840975 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:18.840899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:21.875870 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:21.875840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:25.065442 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:25.065403    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:30.968115 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:30.968076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:31.065771 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:31.065738    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:34.860493 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:34.860448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:35.091020 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.090985    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"]
Aug 05 18:06:35.091339 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.091323    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" containerName="main"
Aug 05 18:06:35.091339 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.091338    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" containerName="main"
Aug 05 18:06:35.091435 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.091352    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerName="main"
Aug 05 18:06:35.091435 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.091357    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerName="main"
Aug 05 18:06:35.091435 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.091430    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff59d1fe-a01e-4f71-9a8e-36c675bd6fcf" containerName="main"
Aug 05 18:06:35.091543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.091439    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f0de14-25a6-41cc-a8f8-12b3c7376c0e" containerName="main"
Aug 05 18:06:35.094324 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.094295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.097964 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.097932    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-kserve-self-signed-certs\""
Aug 05 18:06:35.098082 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.097991    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Aug 05 18:06:35.098146 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.098099    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"default-dockercfg-d4wpz\""
Aug 05 18:06:35.098218 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.098199    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-epp-sa-dockercfg-n222g\""
Aug 05 18:06:35.098286 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.098243    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Aug 05 18:06:35.103143 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.103117    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"]
Aug 05 18:06:35.132118 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.132051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:35.235690 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.235664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:06:35.267438 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.267411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afefe79-eeab-4762-ad8f-35f15d92174f-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.267589 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.267443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8mwb\" (UniqueName: \"kubernetes.io/projected/0afefe79-eeab-4762-ad8f-35f15d92174f-kube-api-access-h8mwb\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.368370 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.368339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afefe79-eeab-4762-ad8f-35f15d92174f-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.368510 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.368375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mwb\" (UniqueName: \"kubernetes.io/projected/0afefe79-eeab-4762-ad8f-35f15d92174f-kube-api-access-h8mwb\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.370992 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.370958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afefe79-eeab-4762-ad8f-35f15d92174f-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.375839 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.375819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8mwb\" (UniqueName: \"kubernetes.io/projected/0afefe79-eeab-4762-ad8f-35f15d92174f-kube-api-access-h8mwb\") pod \"router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.407683 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.407627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:35.535849 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:35.535823    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"]
Aug 05 18:06:35.538540 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:06:35.538515    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0afefe79_eeab_4762_ad8f_35f15d92174f.slice/crio-6f7bb93afd2b798129e928da4843353613a99d3c92a5c215fd7eed280fcf5bf9 WatchSource:0}: Error finding container 6f7bb93afd2b798129e928da4843353613a99d3c92a5c215fd7eed280fcf5bf9: Status 404 returned error can't find the container with id 6f7bb93afd2b798129e928da4843353613a99d3c92a5c215fd7eed280fcf5bf9
Aug 05 18:06:36.248175 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:36.248143    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" event={"ID":"0afefe79-eeab-4762-ad8f-35f15d92174f","Type":"ContainerStarted","Data":"3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8"}
Aug 05 18:06:36.248629 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:36.248192    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" event={"ID":"0afefe79-eeab-4762-ad8f-35f15d92174f","Type":"ContainerStarted","Data":"6f7bb93afd2b798129e928da4843353613a99d3c92a5c215fd7eed280fcf5bf9"}
Aug 05 18:06:36.248629 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:36.248229    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:06:36.264068 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:06:36.264021    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" podStartSLOduration=1.264007321 podStartE2EDuration="1.264007321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:36.263439355 +0000 UTC m=+3254.796623600" watchObservedRunningTime="2026-08-05 18:06:36.264007321 +0000 UTC m=+3254.797191600"
Aug 05 18:07:05.615748 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:05.615711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:07.260408 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:07.260375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:07:22.139020 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:22.138992    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:07:22.144247 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:22.144222    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:07:22.147637 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:22.147610    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:07:22.152036 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:22.152019    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:07:25.720638 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:25.720599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:25.822568 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:25.822535    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:25.920916 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:25.920888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:30.961443 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:30.961414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:32.422987 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:32.422956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:07:32.481188 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:07:32.481157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:23.677633 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:08:23.677596    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:49.803126 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:08:49.803095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:49.814305 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:08:49.814269    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"]
Aug 05 18:08:49.900226 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:08:49.900195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:08:57.260662 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:08:57.260621    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" podUID="0afefe79-eeab-4762-ad8f-35f15d92174f" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:09:04.833421 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:04.833360    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" podUID="0afefe79-eeab-4762-ad8f-35f15d92174f" containerName="main" containerID="cri-o://3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8" gracePeriod=45
Aug 05 18:09:05.085438 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.085377    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:09:05.144143 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.144107    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8mwb\" (UniqueName: \"kubernetes.io/projected/0afefe79-eeab-4762-ad8f-35f15d92174f-kube-api-access-h8mwb\") pod \"0afefe79-eeab-4762-ad8f-35f15d92174f\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") "
Aug 05 18:09:05.144296 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.144205    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afefe79-eeab-4762-ad8f-35f15d92174f-tls-certs\") pod \"0afefe79-eeab-4762-ad8f-35f15d92174f\" (UID: \"0afefe79-eeab-4762-ad8f-35f15d92174f\") "
Aug 05 18:09:05.146326 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.146276    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0afefe79-eeab-4762-ad8f-35f15d92174f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0afefe79-eeab-4762-ad8f-35f15d92174f" (UID: "0afefe79-eeab-4762-ad8f-35f15d92174f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:09:05.146448 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.146353    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0afefe79-eeab-4762-ad8f-35f15d92174f-kube-api-access-h8mwb" (OuterVolumeSpecName: "kube-api-access-h8mwb") pod "0afefe79-eeab-4762-ad8f-35f15d92174f" (UID: "0afefe79-eeab-4762-ad8f-35f15d92174f"). InnerVolumeSpecName "kube-api-access-h8mwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:09:05.245584 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.245553    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0afefe79-eeab-4762-ad8f-35f15d92174f-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:09:05.245584 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.245578    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8mwb\" (UniqueName: \"kubernetes.io/projected/0afefe79-eeab-4762-ad8f-35f15d92174f-kube-api-access-h8mwb\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:09:05.943368 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.943305    2562 generic.go:356] "Generic (PLEG): container finished" podID="0afefe79-eeab-4762-ad8f-35f15d92174f" containerID="3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8" exitCode=0
Aug 05 18:09:05.943841 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.943371    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"
Aug 05 18:09:05.943841 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.943344    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" event={"ID":"0afefe79-eeab-4762-ad8f-35f15d92174f","Type":"ContainerDied","Data":"3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8"}
Aug 05 18:09:05.943841 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.943472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9" event={"ID":"0afefe79-eeab-4762-ad8f-35f15d92174f","Type":"ContainerDied","Data":"6f7bb93afd2b798129e928da4843353613a99d3c92a5c215fd7eed280fcf5bf9"}
Aug 05 18:09:05.943841 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.943501    2562 scope.go:117] "RemoveContainer" containerID="3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8"
Aug 05 18:09:05.952559 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.952542    2562 scope.go:117] "RemoveContainer" containerID="3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8"
Aug 05 18:09:05.952795 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:09:05.952771    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8\": container with ID starting with 3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8 not found: ID does not exist" containerID="3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8"
Aug 05 18:09:05.952877 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.952804    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8"} err="failed to get container status \"3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8\": rpc error: code = NotFound desc = could not find container \"3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8\": container with ID starting with 3cf25f1ca76244d6168d4066b5d6ace23fd87c6b5f04e019b67cbbd912bf40e8 not found: ID does not exist"
Aug 05 18:09:05.965928 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.965903    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"]
Aug 05 18:09:05.969664 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:05.969643    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-6d9fd8bcc-dx9d9"]
Aug 05 18:09:06.102037 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:06.101989    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0afefe79-eeab-4762-ad8f-35f15d92174f" path="/var/lib/kubelet/pods/0afefe79-eeab-4762-ad8f-35f15d92174f/volumes"
Aug 05 18:09:09.850608 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:09.850577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:09.958061 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:09.958023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:10.068993 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:10.068961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:14.863300 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:14.863222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:23.137032 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.136997    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:23.326002 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.325968    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"]
Aug 05 18:09:23.326414 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.326397    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0afefe79-eeab-4762-ad8f-35f15d92174f" containerName="main"
Aug 05 18:09:23.326511 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.326415    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0afefe79-eeab-4762-ad8f-35f15d92174f" containerName="main"
Aug 05 18:09:23.326511 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.326501    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0afefe79-eeab-4762-ad8f-35f15d92174f" containerName="main"
Aug 05 18:09:23.331390 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.331368    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.335434 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.335405    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Aug 05 18:09:23.335566 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.335472    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-mwxt8\""
Aug 05 18:09:23.335566 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.335529    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Aug 05 18:09:23.335566 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.335529    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Aug 05 18:09:23.335782 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.335687    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-b9k5b\""
Aug 05 18:09:23.338341 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.338298    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"]
Aug 05 18:09:23.347881 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.347862    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:23.403813 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.403734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69481222-bf08-4254-9f84-85aee70b7699-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.403813 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.403776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nm7r\" (UniqueName: \"kubernetes.io/projected/69481222-bf08-4254-9f84-85aee70b7699-kube-api-access-2nm7r\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.504867 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.504826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69481222-bf08-4254-9f84-85aee70b7699-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.505013 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.504886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nm7r\" (UniqueName: \"kubernetes.io/projected/69481222-bf08-4254-9f84-85aee70b7699-kube-api-access-2nm7r\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.507334 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.507284    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69481222-bf08-4254-9f84-85aee70b7699-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.513081 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.513051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nm7r\" (UniqueName: \"kubernetes.io/projected/69481222-bf08-4254-9f84-85aee70b7699-kube-api-access-2nm7r\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.643146 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.643098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:23.772766 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:23.772739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"]
Aug 05 18:09:23.774936 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:09:23.774910    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69481222_bf08_4254_9f84_85aee70b7699.slice/crio-7ee57a6c7f82dd41a81ebd4aa47d33224ea5f23bde923a65c4d384b65f2b8baa WatchSource:0}: Error finding container 7ee57a6c7f82dd41a81ebd4aa47d33224ea5f23bde923a65c4d384b65f2b8baa: Status 404 returned error can't find the container with id 7ee57a6c7f82dd41a81ebd4aa47d33224ea5f23bde923a65c4d384b65f2b8baa
Aug 05 18:09:24.027115 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:24.027071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" event={"ID":"69481222-bf08-4254-9f84-85aee70b7699","Type":"ContainerStarted","Data":"1f24e810fd69be40aaf14b7beccafb98e9c76154519b6f8f0f1010e008f1f163"}
Aug 05 18:09:24.027344 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:24.027129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" event={"ID":"69481222-bf08-4254-9f84-85aee70b7699","Type":"ContainerStarted","Data":"7ee57a6c7f82dd41a81ebd4aa47d33224ea5f23bde923a65c4d384b65f2b8baa"}
Aug 05 18:09:24.027344 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:24.027233    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:09:24.047485 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:24.047434    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" podStartSLOduration=1.04741889 podStartE2EDuration="1.04741889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:09:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:24.045612074 +0000 UTC m=+3422.578796340" watchObservedRunningTime="2026-08-05 18:09:24.04741889 +0000 UTC m=+3422.580603151"
Aug 05 18:09:35.967691 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:35.967633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:36.838103 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:36.838072    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:38.096561 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:38.096526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:40.197432 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:40.197392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:44.442794 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:44.442758    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:09:55.037056 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:09:55.037022    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:10:00.084395 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:00.084361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:00.184656 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:00.184620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:00.284629 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:00.284586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:00.708675 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:00.708648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:06.199531 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:06.199493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:06.302201 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:06.302173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:07.180257 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:07.180226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:07.234385 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:07.234356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:08.127130 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.127095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:08.363944 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.363911    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"]
Aug 05 18:10:08.367084 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.367061    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.370891 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.370869    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-1d9e6e79\"/\"kube-root-ca.crt\""
Aug 05 18:10:08.371024 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.370911    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-1d9e6e79\"/\"openshift-service-ca.crt\""
Aug 05 18:10:08.371024 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.370925    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-1d9e6e79\"/\"tls-verification-test-kserve-self-signed-certs\""
Aug 05 18:10:08.371024 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.370966    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-1d9e6e79\"/\"tls-verification-test-epp-sa-dockercfg-d9dgb\""
Aug 05 18:10:08.371024 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.370911    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-1d9e6e79\"/\"default-dockercfg-mlptb\""
Aug 05 18:10:08.371248 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.371101    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-5585754765-z5t2q\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") " pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.371248 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.371144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tz9f\" (UniqueName: \"kubernetes.io/projected/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-kube-api-access-4tz9f\") pod \"tls-verification-test-kserve-router-scheduler-5585754765-z5t2q\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") " pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.375561 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.375530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"]
Aug 05 18:10:08.385605 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.385545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:08.472078 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.472051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-5585754765-z5t2q\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") " pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.472204 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.472093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tz9f\" (UniqueName: \"kubernetes.io/projected/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-kube-api-access-4tz9f\") pod \"tls-verification-test-kserve-router-scheduler-5585754765-z5t2q\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") " pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.474465 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.474444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-5585754765-z5t2q\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") " pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.479992 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.479972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tz9f\" (UniqueName: \"kubernetes.io/projected/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-kube-api-access-4tz9f\") pod \"tls-verification-test-kserve-router-scheduler-5585754765-z5t2q\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") " pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.678358 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.678257    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:08.805557 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:08.805525    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"]
Aug 05 18:10:08.808268 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:10:08.808241    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51cd2af_6844_450e_bf1e_52ac1cfb56e2.slice/crio-6d923fe31b34ed12aaea628a18cc763fbbce7524d7a1754cc5ee4310324f90c5 WatchSource:0}: Error finding container 6d923fe31b34ed12aaea628a18cc763fbbce7524d7a1754cc5ee4310324f90c5: Status 404 returned error can't find the container with id 6d923fe31b34ed12aaea628a18cc763fbbce7524d7a1754cc5ee4310324f90c5
Aug 05 18:10:09.241425 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:09.241389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" event={"ID":"c51cd2af-6844-450e-bf1e-52ac1cfb56e2","Type":"ContainerStarted","Data":"e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae"}
Aug 05 18:10:09.241591 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:09.241454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" event={"ID":"c51cd2af-6844-450e-bf1e-52ac1cfb56e2","Type":"ContainerStarted","Data":"6d923fe31b34ed12aaea628a18cc763fbbce7524d7a1754cc5ee4310324f90c5"}
Aug 05 18:10:09.241591 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:09.241536    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:10:09.262437 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:09.262377    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" podStartSLOduration=1.262357872 podStartE2EDuration="1.262357872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:10:09.260263802 +0000 UTC m=+3467.793448047" watchObservedRunningTime="2026-08-05 18:10:09.262357872 +0000 UTC m=+3467.795542118"
Aug 05 18:10:22.493564 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:22.493529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:28.387857 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:28.387820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:10:40.247966 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:10:40.247937    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:11:35.455711 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:35.455678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:35.468801 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:35.468768    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"]
Aug 05 18:11:35.554048 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:35.554016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:45.036505 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:45.036469    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" podUID="69481222-bf08-4254-9f84-85aee70b7699" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:11:50.492886 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.492848    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" podUID="69481222-bf08-4254-9f84-85aee70b7699" containerName="main" containerID="cri-o://1f24e810fd69be40aaf14b7beccafb98e9c76154519b6f8f0f1010e008f1f163" gracePeriod=45
Aug 05 18:11:50.719065 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.719038    2562 generic.go:356] "Generic (PLEG): container finished" podID="69481222-bf08-4254-9f84-85aee70b7699" containerID="1f24e810fd69be40aaf14b7beccafb98e9c76154519b6f8f0f1010e008f1f163" exitCode=0
Aug 05 18:11:50.719178 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.719111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" event={"ID":"69481222-bf08-4254-9f84-85aee70b7699","Type":"ContainerDied","Data":"1f24e810fd69be40aaf14b7beccafb98e9c76154519b6f8f0f1010e008f1f163"}
Aug 05 18:11:50.737815 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.737791    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:11:50.889760 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.889691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nm7r\" (UniqueName: \"kubernetes.io/projected/69481222-bf08-4254-9f84-85aee70b7699-kube-api-access-2nm7r\") pod \"69481222-bf08-4254-9f84-85aee70b7699\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") "
Aug 05 18:11:50.889760 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.889754    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69481222-bf08-4254-9f84-85aee70b7699-tls-certs\") pod \"69481222-bf08-4254-9f84-85aee70b7699\" (UID: \"69481222-bf08-4254-9f84-85aee70b7699\") "
Aug 05 18:11:50.891885 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.891858    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69481222-bf08-4254-9f84-85aee70b7699-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69481222-bf08-4254-9f84-85aee70b7699" (UID: "69481222-bf08-4254-9f84-85aee70b7699"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:11:50.891990 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.891950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69481222-bf08-4254-9f84-85aee70b7699-kube-api-access-2nm7r" (OuterVolumeSpecName: "kube-api-access-2nm7r") pod "69481222-bf08-4254-9f84-85aee70b7699" (UID: "69481222-bf08-4254-9f84-85aee70b7699"). InnerVolumeSpecName "kube-api-access-2nm7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:11:50.990848 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.990823    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nm7r\" (UniqueName: \"kubernetes.io/projected/69481222-bf08-4254-9f84-85aee70b7699-kube-api-access-2nm7r\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:11:50.990848 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:50.990846    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69481222-bf08-4254-9f84-85aee70b7699-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.724529 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:51.724494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7" event={"ID":"69481222-bf08-4254-9f84-85aee70b7699","Type":"ContainerDied","Data":"7ee57a6c7f82dd41a81ebd4aa47d33224ea5f23bde923a65c4d384b65f2b8baa"}
Aug 05 18:11:51.724529 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:51.724513    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"
Aug 05 18:11:51.725061 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:51.724552    2562 scope.go:117] "RemoveContainer" containerID="1f24e810fd69be40aaf14b7beccafb98e9c76154519b6f8f0f1010e008f1f163"
Aug 05 18:11:51.746748 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:51.746715    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"]
Aug 05 18:11:51.750326 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:51.750291    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schem4hn7"]
Aug 05 18:11:52.102781 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:52.102700    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69481222-bf08-4254-9f84-85aee70b7699" path="/var/lib/kubelet/pods/69481222-bf08-4254-9f84-85aee70b7699/volumes"
Aug 05 18:11:55.419375 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:55.419341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:55.518116 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:55.518083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:57.554324 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:57.554290    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:11:57.608930 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:11:57.608901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:01.731735 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:01.731703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:07.426759 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:07.426726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:07.527177 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:07.527146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:09.412580 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.412545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:09.582415 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.582349    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"]
Aug 05 18:12:09.583001 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.582973    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69481222-bf08-4254-9f84-85aee70b7699" containerName="main"
Aug 05 18:12:09.583001 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.582999    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69481222-bf08-4254-9f84-85aee70b7699" containerName="main"
Aug 05 18:12:09.583221 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.583087    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="69481222-bf08-4254-9f84-85aee70b7699" containerName="main"
Aug 05 18:12:09.586034 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.586014    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.588428 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.588404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Aug 05 18:12:09.588565 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.588439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Aug 05 18:12:09.589599 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.589573    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-fqmzr\""
Aug 05 18:12:09.589720 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.589664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-ppb4w\""
Aug 05 18:12:09.589720 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.589682    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Aug 05 18:12:09.596530 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.596508    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"]
Aug 05 18:12:09.614137 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.614109    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:09.738096 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.738070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e523c04-5e98-4b50-9fe1-b46bf2175c30-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.738230 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.738137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77hlh\" (UniqueName: \"kubernetes.io/projected/4e523c04-5e98-4b50-9fe1-b46bf2175c30-kube-api-access-77hlh\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.839397 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.839360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77hlh\" (UniqueName: \"kubernetes.io/projected/4e523c04-5e98-4b50-9fe1-b46bf2175c30-kube-api-access-77hlh\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.839574 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.839447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e523c04-5e98-4b50-9fe1-b46bf2175c30-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.841933 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.841908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e523c04-5e98-4b50-9fe1-b46bf2175c30-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.848566 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.848534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77hlh\" (UniqueName: \"kubernetes.io/projected/4e523c04-5e98-4b50-9fe1-b46bf2175c30-kube-api-access-77hlh\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:09.897295 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:09.897269    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:10.029481 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:10.029454    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"]
Aug 05 18:12:10.032540 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:12:10.032513    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e523c04_5e98_4b50_9fe1_b46bf2175c30.slice/crio-1a088ad13a9e519163b90283f64fa7dee274ee7aaf7d082b246117001157e9dc WatchSource:0}: Error finding container 1a088ad13a9e519163b90283f64fa7dee274ee7aaf7d082b246117001157e9dc: Status 404 returned error can't find the container with id 1a088ad13a9e519163b90283f64fa7dee274ee7aaf7d082b246117001157e9dc
Aug 05 18:12:10.814992 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:10.814958    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" event={"ID":"4e523c04-5e98-4b50-9fe1-b46bf2175c30","Type":"ContainerStarted","Data":"a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962"}
Aug 05 18:12:10.815425 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:10.815009    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" event={"ID":"4e523c04-5e98-4b50-9fe1-b46bf2175c30","Type":"ContainerStarted","Data":"1a088ad13a9e519163b90283f64fa7dee274ee7aaf7d082b246117001157e9dc"}
Aug 05 18:12:10.815425 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:10.815097    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:10.832596 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:10.832552    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" podStartSLOduration=1.832540266 podStartE2EDuration="1.832540266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:12:10.831011388 +0000 UTC m=+3589.364195634" watchObservedRunningTime="2026-08-05 18:12:10.832540266 +0000 UTC m=+3589.365724520"
Aug 05 18:12:22.168081 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:22.168050    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:12:22.172771 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:22.172744    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:12:22.177772 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:22.177755    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:12:22.182419 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:22.182403    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:12:29.711775 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:29.711743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:33.777234 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:33.777193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:35.610169 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:35.610131    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"]
Aug 05 18:12:35.615430 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:35.615398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:35.675558 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:35.675534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:40.247541 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:40.247506    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:12:41.822430 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:41.822396    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:12:50.247705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.247668    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:12:50.630122 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.630043    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerName="main" containerID="cri-o://e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae" gracePeriod=45
Aug 05 18:12:50.879226 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.879194    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:12:50.975218 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.975178    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-tls-certs\") pod \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") "
Aug 05 18:12:50.975390 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.975246    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tz9f\" (UniqueName: \"kubernetes.io/projected/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-kube-api-access-4tz9f\") pod \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\" (UID: \"c51cd2af-6844-450e-bf1e-52ac1cfb56e2\") "
Aug 05 18:12:50.977427 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.977403    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c51cd2af-6844-450e-bf1e-52ac1cfb56e2" (UID: "c51cd2af-6844-450e-bf1e-52ac1cfb56e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:12:50.977532 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:50.977483    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-kube-api-access-4tz9f" (OuterVolumeSpecName: "kube-api-access-4tz9f") pod "c51cd2af-6844-450e-bf1e-52ac1cfb56e2" (UID: "c51cd2af-6844-450e-bf1e-52ac1cfb56e2"). InnerVolumeSpecName "kube-api-access-4tz9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:12:51.003700 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.003673    2562 generic.go:356] "Generic (PLEG): container finished" podID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerID="e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae" exitCode=0
Aug 05 18:12:51.003783 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.003716    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"
Aug 05 18:12:51.003783 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.003758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" event={"ID":"c51cd2af-6844-450e-bf1e-52ac1cfb56e2","Type":"ContainerDied","Data":"e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae"}
Aug 05 18:12:51.003867 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.003805    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q" event={"ID":"c51cd2af-6844-450e-bf1e-52ac1cfb56e2","Type":"ContainerDied","Data":"6d923fe31b34ed12aaea628a18cc763fbbce7524d7a1754cc5ee4310324f90c5"}
Aug 05 18:12:51.003867 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.003830    2562 scope.go:117] "RemoveContainer" containerID="e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae"
Aug 05 18:12:51.012175 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.012156    2562 scope.go:117] "RemoveContainer" containerID="e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae"
Aug 05 18:12:51.012468 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:12:51.012446    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae\": container with ID starting with e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae not found: ID does not exist" containerID="e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae"
Aug 05 18:12:51.012540 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.012481    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae"} err="failed to get container status \"e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae\": rpc error: code = NotFound desc = could not find container \"e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae\": container with ID starting with e5b6d69ad6f45505abd680d6a2d3e565c2b13ada1ba77232ca246bcb86b65dae not found: ID does not exist"
Aug 05 18:12:51.024575 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.024551    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"]
Aug 05 18:12:51.028008 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.027987    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-tls-resources-1d9e6e79/tls-verification-test-kserve-router-scheduler-5585754765-z5t2q"]
Aug 05 18:12:51.076599 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.076575    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tz9f\" (UniqueName: \"kubernetes.io/projected/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-kube-api-access-4tz9f\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:12:51.076599 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:51.076596    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c51cd2af-6844-450e-bf1e-52ac1cfb56e2-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:12:52.102544 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:52.102514    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" path="/var/lib/kubelet/pods/c51cd2af-6844-450e-bf1e-52ac1cfb56e2/volumes"
Aug 05 18:12:52.989252 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:52.989217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:53.083216 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:53.083184    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:12:59.170039 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:12:59.170012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:03.567991 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.567960    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:03.714809 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.714776    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"]
Aug 05 18:13:03.715141 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.715126    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerName="main"
Aug 05 18:13:03.715141 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.715142    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerName="main"
Aug 05 18:13:03.715279 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.715209    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51cd2af-6844-450e-bf1e-52ac1cfb56e2" containerName="main"
Aug 05 18:13:03.719648 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.719625    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:03.721973 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.721948    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Aug 05 18:13:03.722096 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.722008    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Aug 05 18:13:03.722096 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.722019    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-epp-sa-dockercfg-8sdd6\""
Aug 05 18:13:03.723113 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.723095    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Aug 05 18:13:03.723203 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.723095    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-n6n48\""
Aug 05 18:13:03.726100 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.726076    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"]
Aug 05 18:13:03.738431 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.738410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:03.879697 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.879613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvrdw\" (UniqueName: \"kubernetes.io/projected/a889c2b6-e38e-4860-ad5b-f7093da73447-kube-api-access-wvrdw\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:03.879833 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.879724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a889c2b6-e38e-4860-ad5b-f7093da73447-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:03.980700 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.980657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvrdw\" (UniqueName: \"kubernetes.io/projected/a889c2b6-e38e-4860-ad5b-f7093da73447-kube-api-access-wvrdw\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:03.980855 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.980736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a889c2b6-e38e-4860-ad5b-f7093da73447-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:03.983047 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.983020    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a889c2b6-e38e-4860-ad5b-f7093da73447-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:03.988407 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:03.988385    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvrdw\" (UniqueName: \"kubernetes.io/projected/a889c2b6-e38e-4860-ad5b-f7093da73447-kube-api-access-wvrdw\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:04.031122 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:04.031098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:04.166803 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:04.166720    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"]
Aug 05 18:13:04.170594 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:13:04.170564    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda889c2b6_e38e_4860_ad5b_f7093da73447.slice/crio-3db1f900cc42eae056212ffe7113329b2c5a7e5e6b9097df63b491d90f8c7f85 WatchSource:0}: Error finding container 3db1f900cc42eae056212ffe7113329b2c5a7e5e6b9097df63b491d90f8c7f85: Status 404 returned error can't find the container with id 3db1f900cc42eae056212ffe7113329b2c5a7e5e6b9097df63b491d90f8c7f85
Aug 05 18:13:05.069103 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:05.069061    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" event={"ID":"a889c2b6-e38e-4860-ad5b-f7093da73447","Type":"ContainerStarted","Data":"f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792"}
Aug 05 18:13:05.069491 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:05.069129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" event={"ID":"a889c2b6-e38e-4860-ad5b-f7093da73447","Type":"ContainerStarted","Data":"3db1f900cc42eae056212ffe7113329b2c5a7e5e6b9097df63b491d90f8c7f85"}
Aug 05 18:13:05.069491 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:05.069185    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:13:05.085776 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:05.085732    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" podStartSLOduration=2.085718794 podStartE2EDuration="2.085718794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:13:05.084307958 +0000 UTC m=+3643.617492205" watchObservedRunningTime="2026-08-05 18:13:05.085718794 +0000 UTC m=+3643.618903038"
Aug 05 18:13:05.369972 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:05.369889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:05.476073 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:05.476037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:17.190294 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:17.190263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:21.965844 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:21.965815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:13:36.077246 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:13:36.077214    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:14:19.059440 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:19.059402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:19.060983 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:19.060955    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"]
Aug 05 18:14:19.153634 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:19.153606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:21.822656 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:21.822624    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:14:31.821707 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:31.821664    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:14:34.080334 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.080261    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerName="main" containerID="cri-o://a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962" gracePeriod=45
Aug 05 18:14:34.330331 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.330295    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:14:34.405931 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.405861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77hlh\" (UniqueName: \"kubernetes.io/projected/4e523c04-5e98-4b50-9fe1-b46bf2175c30-kube-api-access-77hlh\") pod \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") "
Aug 05 18:14:34.405931 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.405900    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e523c04-5e98-4b50-9fe1-b46bf2175c30-tls-certs\") pod \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\" (UID: \"4e523c04-5e98-4b50-9fe1-b46bf2175c30\") "
Aug 05 18:14:34.408045 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.408010    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e523c04-5e98-4b50-9fe1-b46bf2175c30-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e523c04-5e98-4b50-9fe1-b46bf2175c30" (UID: "4e523c04-5e98-4b50-9fe1-b46bf2175c30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:14:34.408153 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.408058    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e523c04-5e98-4b50-9fe1-b46bf2175c30-kube-api-access-77hlh" (OuterVolumeSpecName: "kube-api-access-77hlh") pod "4e523c04-5e98-4b50-9fe1-b46bf2175c30" (UID: "4e523c04-5e98-4b50-9fe1-b46bf2175c30"). InnerVolumeSpecName "kube-api-access-77hlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:14:34.493608 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.493577    2562 generic.go:356] "Generic (PLEG): container finished" podID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerID="a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962" exitCode=0
Aug 05 18:14:34.493768 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.493623    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"
Aug 05 18:14:34.493768 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.493662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" event={"ID":"4e523c04-5e98-4b50-9fe1-b46bf2175c30","Type":"ContainerDied","Data":"a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962"}
Aug 05 18:14:34.493768 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.493712    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm" event={"ID":"4e523c04-5e98-4b50-9fe1-b46bf2175c30","Type":"ContainerDied","Data":"1a088ad13a9e519163b90283f64fa7dee274ee7aaf7d082b246117001157e9dc"}
Aug 05 18:14:34.493768 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.493740    2562 scope.go:117] "RemoveContainer" containerID="a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962"
Aug 05 18:14:34.504610 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.504592    2562 scope.go:117] "RemoveContainer" containerID="a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962"
Aug 05 18:14:34.504884 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:14:34.504865    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962\": container with ID starting with a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962 not found: ID does not exist" containerID="a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962"
Aug 05 18:14:34.504932 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.504893    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962"} err="failed to get container status \"a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962\": rpc error: code = NotFound desc = could not find container \"a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962\": container with ID starting with a38921f9a38856e072d1817cf573dc26f908a0c05472f7b72681320c4788f962 not found: ID does not exist"
Aug 05 18:14:34.507269 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.507248    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77hlh\" (UniqueName: \"kubernetes.io/projected/4e523c04-5e98-4b50-9fe1-b46bf2175c30-kube-api-access-77hlh\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:14:34.507417 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.507272    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e523c04-5e98-4b50-9fe1-b46bf2175c30-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:14:34.516754 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.516731    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"]
Aug 05 18:14:34.520474 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:34.520452    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-74d74hztwm"]
Aug 05 18:14:36.102474 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:36.102435    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" path="/var/lib/kubelet/pods/4e523c04-5e98-4b50-9fe1-b46bf2175c30/volumes"
Aug 05 18:14:39.161095 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:39.161059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:39.251289 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:39.251260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:39.408538 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:39.408502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:39.474046 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:39.474018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:42.859702 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:42.859673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:45.836985 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:45.836947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:51.662358 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:51.662304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:51.722377 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:51.722347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:56.955427 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:56.955385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:57.187490 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.187452    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"]
Aug 05 18:14:57.187844 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.187828    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerName="main"
Aug 05 18:14:57.187844 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.187843    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerName="main"
Aug 05 18:14:57.187988 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.187919    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e523c04-5e98-4b50-9fe1-b46bf2175c30" containerName="main"
Aug 05 18:14:57.191086 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.191064    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.193855 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.193826    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Aug 05 18:14:57.194885 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.194860    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-5k8p2\""
Aug 05 18:14:57.195009 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.194913    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Aug 05 18:14:57.195009 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.194936    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Aug 05 18:14:57.195009 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.194970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-4xrgk\""
Aug 05 18:14:57.199484 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.199164    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"]
Aug 05 18:14:57.224629 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.224568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:57.297009 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.296980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27d889e0-9496-4c92-a859-e5237dbda52c-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.297150 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.297029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkr7g\" (UniqueName: \"kubernetes.io/projected/27d889e0-9496-4c92-a859-e5237dbda52c-kube-api-access-zkr7g\") pod \"router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.306449 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.306425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:14:57.397756 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.397722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27d889e0-9496-4c92-a859-e5237dbda52c-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.397756 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.397762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkr7g\" (UniqueName: \"kubernetes.io/projected/27d889e0-9496-4c92-a859-e5237dbda52c-kube-api-access-zkr7g\") pod \"router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.400297 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.400276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27d889e0-9496-4c92-a859-e5237dbda52c-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.404813 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.404787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkr7g\" (UniqueName: \"kubernetes.io/projected/27d889e0-9496-4c92-a859-e5237dbda52c-kube-api-access-zkr7g\") pod \"router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.505091 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.505005    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:57.634260 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:57.634236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"]
Aug 05 18:14:57.636867 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:14:57.636843    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d889e0_9496_4c92_a859_e5237dbda52c.slice/crio-9ea0a358537f83bbd1fd356b99d2ada77ab35ebfe2a7afcfdccb89acabbd3747 WatchSource:0}: Error finding container 9ea0a358537f83bbd1fd356b99d2ada77ab35ebfe2a7afcfdccb89acabbd3747: Status 404 returned error can't find the container with id 9ea0a358537f83bbd1fd356b99d2ada77ab35ebfe2a7afcfdccb89acabbd3747
Aug 05 18:14:58.607384 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:58.607340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" event={"ID":"27d889e0-9496-4c92-a859-e5237dbda52c","Type":"ContainerStarted","Data":"49e5b7344b9252efe666e61987344b091bbb08389c6a7a7ae337e351350b5917"}
Aug 05 18:14:58.607858 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:58.607412    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" event={"ID":"27d889e0-9496-4c92-a859-e5237dbda52c","Type":"ContainerStarted","Data":"9ea0a358537f83bbd1fd356b99d2ada77ab35ebfe2a7afcfdccb89acabbd3747"}
Aug 05 18:14:58.607858 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:58.607453    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:14:58.623928 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:14:58.623882    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" podStartSLOduration=1.623869695 podStartE2EDuration="1.623869695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:58.621153445 +0000 UTC m=+3757.154337691" watchObservedRunningTime="2026-08-05 18:14:58.623869695 +0000 UTC m=+3757.157053956"
Aug 05 18:15:27.015162 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:27.015131    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"]
Aug 05 18:15:27.028587 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:27.028549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:27.090924 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:27.090898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:15:29.616755 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:29.616729    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:15:36.076872 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:36.076830    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" podUID="a889c2b6-e38e-4860-ad5b-f7093da73447" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:15:42.035155 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.035110    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" podUID="a889c2b6-e38e-4860-ad5b-f7093da73447" containerName="main" containerID="cri-o://f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792" gracePeriod=45
Aug 05 18:15:42.284843 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.284814    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:15:42.360058 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.359982    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvrdw\" (UniqueName: \"kubernetes.io/projected/a889c2b6-e38e-4860-ad5b-f7093da73447-kube-api-access-wvrdw\") pod \"a889c2b6-e38e-4860-ad5b-f7093da73447\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") "
Aug 05 18:15:42.360058 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.360048    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a889c2b6-e38e-4860-ad5b-f7093da73447-tls-certs\") pod \"a889c2b6-e38e-4860-ad5b-f7093da73447\" (UID: \"a889c2b6-e38e-4860-ad5b-f7093da73447\") "
Aug 05 18:15:42.362247 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.362216    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a889c2b6-e38e-4860-ad5b-f7093da73447-kube-api-access-wvrdw" (OuterVolumeSpecName: "kube-api-access-wvrdw") pod "a889c2b6-e38e-4860-ad5b-f7093da73447" (UID: "a889c2b6-e38e-4860-ad5b-f7093da73447"). InnerVolumeSpecName "kube-api-access-wvrdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:15:42.362380 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.362237    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a889c2b6-e38e-4860-ad5b-f7093da73447-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a889c2b6-e38e-4860-ad5b-f7093da73447" (UID: "a889c2b6-e38e-4860-ad5b-f7093da73447"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:15:42.460792 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.460763    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvrdw\" (UniqueName: \"kubernetes.io/projected/a889c2b6-e38e-4860-ad5b-f7093da73447-kube-api-access-wvrdw\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:15:42.460792 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.460786    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a889c2b6-e38e-4860-ad5b-f7093da73447-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:15:42.812256 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.812217    2562 generic.go:356] "Generic (PLEG): container finished" podID="a889c2b6-e38e-4860-ad5b-f7093da73447" containerID="f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792" exitCode=0
Aug 05 18:15:42.812458 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.812265    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"
Aug 05 18:15:42.812458 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.812288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" event={"ID":"a889c2b6-e38e-4860-ad5b-f7093da73447","Type":"ContainerDied","Data":"f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792"}
Aug 05 18:15:42.812458 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.812367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc" event={"ID":"a889c2b6-e38e-4860-ad5b-f7093da73447","Type":"ContainerDied","Data":"3db1f900cc42eae056212ffe7113329b2c5a7e5e6b9097df63b491d90f8c7f85"}
Aug 05 18:15:42.812458 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.812398    2562 scope.go:117] "RemoveContainer" containerID="f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792"
Aug 05 18:15:42.821305 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.821286    2562 scope.go:117] "RemoveContainer" containerID="f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792"
Aug 05 18:15:42.821579 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:15:42.821560    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792\": container with ID starting with f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792 not found: ID does not exist" containerID="f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792"
Aug 05 18:15:42.821626 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.821587    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792"} err="failed to get container status \"f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792\": rpc error: code = NotFound desc = could not find container \"f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792\": container with ID starting with f90581b4ae2561441ae56b5644db4ca1a2e2650746afae3407da4d7402796792 not found: ID does not exist"
Aug 05 18:15:42.834254 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.834232    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"]
Aug 05 18:15:42.838385 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:42.838366    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-ndcdc"]
Aug 05 18:15:44.102687 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:15:44.102650    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a889c2b6-e38e-4860-ad5b-f7093da73447" path="/var/lib/kubelet/pods/a889c2b6-e38e-4860-ad5b-f7093da73447/volumes"
Aug 05 18:16:32.134881 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:32.134833    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:32.238721 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:32.238695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:38.559202 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:38.559169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:44.286119 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:44.286084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:44.386288 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:44.386259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:46.220513 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.220477    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"]
Aug 05 18:16:46.220911 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.220827    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a889c2b6-e38e-4860-ad5b-f7093da73447" containerName="main"
Aug 05 18:16:46.220911 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.220838    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a889c2b6-e38e-4860-ad5b-f7093da73447" containerName="main"
Aug 05 18:16:46.220911 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.220900    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a889c2b6-e38e-4860-ad5b-f7093da73447" containerName="main"
Aug 05 18:16:46.223531 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.223512    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.228174 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.228145    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Aug 05 18:16:46.232330 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.228451    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Aug 05 18:16:46.232330 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.228764    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Aug 05 18:16:46.232330 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.229049    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-5r4pk\""
Aug 05 18:16:46.237536 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.237514    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"]
Aug 05 18:16:46.266454 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.266429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.266554 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.266469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.266554 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.266526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzhbl\" (UniqueName: \"kubernetes.io/projected/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-kube-api-access-gzhbl\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.266631 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.266560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.266631 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.266588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.266697 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.266633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.367672 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.367639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.367806 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.367680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.367806 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.367715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzhbl\" (UniqueName: \"kubernetes.io/projected/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-kube-api-access-gzhbl\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.367806 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.367746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.367806 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.367768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.368103 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.368079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.368173 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.367797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.368275 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.368251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.368445 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.368398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.370572 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.370546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.370855 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.370831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.376173 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.376151    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzhbl\" (UniqueName: \"kubernetes.io/projected/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-kube-api-access-gzhbl\") pod \"rolling-upgrade-test-kserve-74b79d7f77-9nt54\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.411788 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.411760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:46.541118 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.541053    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:46.671596 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.671570    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"]
Aug 05 18:16:46.673509 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:16:46.673484    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda495ed9b_7728_4ed1_b3c6_d3effcf0a6d3.slice/crio-95fa51c9972545d0e3ff5c0dcb411489cd7405f14439b445338a74b2a44818d3 WatchSource:0}: Error finding container 95fa51c9972545d0e3ff5c0dcb411489cd7405f14439b445338a74b2a44818d3: Status 404 returned error can't find the container with id 95fa51c9972545d0e3ff5c0dcb411489cd7405f14439b445338a74b2a44818d3
Aug 05 18:16:46.758309 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:46.758278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:47.117151 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:47.117117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" event={"ID":"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3","Type":"ContainerStarted","Data":"3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"}
Aug 05 18:16:47.117382 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:47.117168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" event={"ID":"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3","Type":"ContainerStarted","Data":"95fa51c9972545d0e3ff5c0dcb411489cd7405f14439b445338a74b2a44818d3"}
Aug 05 18:16:47.132063 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:47.132013    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:16:47.135501 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:47.135452    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" podStartSLOduration=1.135435524 podStartE2EDuration="1.135435524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:16:47.134504026 +0000 UTC m=+3865.667688271" watchObservedRunningTime="2026-08-05 18:16:47.135435524 +0000 UTC m=+3865.668619768"
Aug 05 18:16:47.541998 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:47.541961    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:56.541995 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:56.541960    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:56.554304 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:56.554276    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:16:57.183133 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:16:57.183104    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:17:00.572213 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:00.572183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:11.046359 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:11.046310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:11.059738 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:11.059708    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"]
Aug 05 18:17:11.101854 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:11.101825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:12.750689 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:12.750654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:19.616622 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:19.616580    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" podUID="27d889e0-9496-4c92-a859-e5237dbda52c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Aug 05 18:17:20.000307 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.000270    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"]
Aug 05 18:17:20.003812 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.003792    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.010760 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.010731    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"]
Aug 05 18:17:20.165271 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.165237    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.165271 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.165276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5933cc73-3629-44f7-a23e-ee3d88675813-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.165543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.165381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.165543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.165408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.165543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.165436    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.165543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.165455    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrjxs\" (UniqueName: \"kubernetes.io/projected/5933cc73-3629-44f7-a23e-ee3d88675813-kube-api-access-zrjxs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.266543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.266462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.266543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.266498    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5933cc73-3629-44f7-a23e-ee3d88675813-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.266724 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.266551    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.266724 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.266577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.266724 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.266689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.266724 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.266716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrjxs\" (UniqueName: \"kubernetes.io/projected/5933cc73-3629-44f7-a23e-ee3d88675813-kube-api-access-zrjxs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.267142 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.267113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-tmp-dir\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.267280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.267113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-model-cache\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.267280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.267193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-home\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.268626 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.268605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-dshm\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.269005 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.268989    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5933cc73-3629-44f7-a23e-ee3d88675813-tls-certs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.275213 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.275191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrjxs\" (UniqueName: \"kubernetes.io/projected/5933cc73-3629-44f7-a23e-ee3d88675813-kube-api-access-zrjxs\") pod \"rolling-upgrade-test-kserve-6dd8dbf85-jn2vf\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.315480 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.315450    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:20.444155 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:20.444126    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"]
Aug 05 18:17:20.446252 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:17:20.446226    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5933cc73_3629_44f7_a23e_ee3d88675813.slice/crio-c46b790ff56bfdf9891d15d30f01a18860b142ed4473a7fa4963535685282c5a WatchSource:0}: Error finding container c46b790ff56bfdf9891d15d30f01a18860b142ed4473a7fa4963535685282c5a: Status 404 returned error can't find the container with id c46b790ff56bfdf9891d15d30f01a18860b142ed4473a7fa4963535685282c5a
Aug 05 18:17:21.283325 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:21.283275    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" event={"ID":"5933cc73-3629-44f7-a23e-ee3d88675813","Type":"ContainerStarted","Data":"dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"}
Aug 05 18:17:21.283716 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:21.283348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" event={"ID":"5933cc73-3629-44f7-a23e-ee3d88675813","Type":"ContainerStarted","Data":"c46b790ff56bfdf9891d15d30f01a18860b142ed4473a7fa4963535685282c5a"}
Aug 05 18:17:21.301060 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:21.301017    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" podStartSLOduration=2.301003858 podStartE2EDuration="2.301003858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:17:21.300356365 +0000 UTC m=+3899.833540611" watchObservedRunningTime="2026-08-05 18:17:21.301003858 +0000 UTC m=+3899.834188104"
Aug 05 18:17:21.315802 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:21.315781    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:22.198163 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:22.198133    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:17:22.202967 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:22.202940    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:17:22.209281 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:22.209262    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:17:22.214104 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:22.214082    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mc8cp_ad6ecaeb-46b2-4555-92f1-26d2f7dff5eb/ovn-acl-logging/0.log"
Aug 05 18:17:24.216552 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.216522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:24.225733 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.225706    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"]
Aug 05 18:17:24.229948 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.229923    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"]
Aug 05 18:17:24.235636 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:17:24.235597    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 18:17:24.235636 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:17:24.235636 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:17:24.235636 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b" cmd=["/bin/sleep","15"]
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.235652    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" message="" err=<
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:17:24.235666    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" podUID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" containerName="main" containerID="cri-o://3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"
Aug 05 18:17:24.235917 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.235699    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" podUID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" containerName="main" containerID="cri-o://3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b" gracePeriod=60
Aug 05 18:17:24.306861 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:17:24.306821    2562 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 18:17:24.306861 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:17:24.306861 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:17:24.306861 ip-10-0-131-218 kubenswrapper[2562]:  > containerID="dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80" cmd=["/bin/sleep","15"]
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.306868    2562 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" message="" err=<
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]:  >
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:17:24.306881    2562 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]:         , stdout: , stderr: , exit code -1
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" podUID="5933cc73-3629-44f7-a23e-ee3d88675813" containerName="main" containerID="cri-o://dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"
Aug 05 18:17:24.307105 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.306912    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" podUID="5933cc73-3629-44f7-a23e-ee3d88675813" containerName="main" containerID="cri-o://dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80" gracePeriod=60
Aug 05 18:17:24.321796 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.321773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:24.575957 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.575933    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:24.600641 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600616    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-model-cache\") pod \"5933cc73-3629-44f7-a23e-ee3d88675813\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") "
Aug 05 18:17:24.600782 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600682    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrjxs\" (UniqueName: \"kubernetes.io/projected/5933cc73-3629-44f7-a23e-ee3d88675813-kube-api-access-zrjxs\") pod \"5933cc73-3629-44f7-a23e-ee3d88675813\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") "
Aug 05 18:17:24.600782 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600763    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-dshm\") pod \"5933cc73-3629-44f7-a23e-ee3d88675813\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") "
Aug 05 18:17:24.600888 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600794    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-home\") pod \"5933cc73-3629-44f7-a23e-ee3d88675813\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") "
Aug 05 18:17:24.600888 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5933cc73-3629-44f7-a23e-ee3d88675813-tls-certs\") pod \"5933cc73-3629-44f7-a23e-ee3d88675813\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") "
Aug 05 18:17:24.600888 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-model-cache" (OuterVolumeSpecName: "model-cache") pod "5933cc73-3629-44f7-a23e-ee3d88675813" (UID: "5933cc73-3629-44f7-a23e-ee3d88675813"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.601025 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.600898    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-tmp-dir\") pod \"5933cc73-3629-44f7-a23e-ee3d88675813\" (UID: \"5933cc73-3629-44f7-a23e-ee3d88675813\") "
Aug 05 18:17:24.601242 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.601081    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-home" (OuterVolumeSpecName: "home") pod "5933cc73-3629-44f7-a23e-ee3d88675813" (UID: "5933cc73-3629-44f7-a23e-ee3d88675813"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.601242 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.601177    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.601242 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.601193    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.601491 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.601286    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5933cc73-3629-44f7-a23e-ee3d88675813" (UID: "5933cc73-3629-44f7-a23e-ee3d88675813"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.603141 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.603116    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5933cc73-3629-44f7-a23e-ee3d88675813-kube-api-access-zrjxs" (OuterVolumeSpecName: "kube-api-access-zrjxs") pod "5933cc73-3629-44f7-a23e-ee3d88675813" (UID: "5933cc73-3629-44f7-a23e-ee3d88675813"). InnerVolumeSpecName "kube-api-access-zrjxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:24.603251 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.603221    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-dshm" (OuterVolumeSpecName: "dshm") pod "5933cc73-3629-44f7-a23e-ee3d88675813" (UID: "5933cc73-3629-44f7-a23e-ee3d88675813"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.603304 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.603277    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5933cc73-3629-44f7-a23e-ee3d88675813-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5933cc73-3629-44f7-a23e-ee3d88675813" (UID: "5933cc73-3629-44f7-a23e-ee3d88675813"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:24.605129 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.605110    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:17:24.702387 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.702349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tls-certs\") pod \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") "
Aug 05 18:17:24.702586 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.702398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzhbl\" (UniqueName: \"kubernetes.io/projected/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-kube-api-access-gzhbl\") pod \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") "
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.702861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-home\") pod \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") "
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703129    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-home" (OuterVolumeSpecName: "home") pod "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" (UID: "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-model-cache\") pod \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") "
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703361    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-dshm\") pod \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") "
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703407    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tmp-dir\") pod \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\" (UID: \"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3\") "
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703825    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703844    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703858    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5933cc73-3629-44f7-a23e-ee3d88675813-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703872    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5933cc73-3629-44f7-a23e-ee3d88675813-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.703892    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrjxs\" (UniqueName: \"kubernetes.io/projected/5933cc73-3629-44f7-a23e-ee3d88675813-kube-api-access-zrjxs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.704219    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" (UID: "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.704705 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.704507    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" (UID: "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.705485 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.705464    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" (UID: "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:24.710132 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.709988    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-dshm" (OuterVolumeSpecName: "dshm") pod "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" (UID: "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:24.710132 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.709993    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-kube-api-access-gzhbl" (OuterVolumeSpecName: "kube-api-access-gzhbl") pod "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" (UID: "a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3"). InnerVolumeSpecName "kube-api-access-gzhbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:24.804604 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.804526    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.804604 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.804555    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.804604 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.804563    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.804604 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.804571    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:24.804604 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:24.804580    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzhbl\" (UniqueName: \"kubernetes.io/projected/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3-kube-api-access-gzhbl\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:25.304472 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.304441    2562 generic.go:356] "Generic (PLEG): container finished" podID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" containerID="3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b" exitCode=0
Aug 05 18:17:25.304928 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.304500    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"
Aug 05 18:17:25.304928 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.304530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" event={"ID":"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3","Type":"ContainerDied","Data":"3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"}
Aug 05 18:17:25.304928 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.304586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54" event={"ID":"a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3","Type":"ContainerDied","Data":"95fa51c9972545d0e3ff5c0dcb411489cd7405f14439b445338a74b2a44818d3"}
Aug 05 18:17:25.304928 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.304604    2562 scope.go:117] "RemoveContainer" containerID="3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"
Aug 05 18:17:25.305967 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.305946    2562 generic.go:356] "Generic (PLEG): container finished" podID="5933cc73-3629-44f7-a23e-ee3d88675813" containerID="dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80" exitCode=0
Aug 05 18:17:25.306071 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.306004    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"
Aug 05 18:17:25.306136 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.306007    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" event={"ID":"5933cc73-3629-44f7-a23e-ee3d88675813","Type":"ContainerDied","Data":"dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"}
Aug 05 18:17:25.306136 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.306124    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf" event={"ID":"5933cc73-3629-44f7-a23e-ee3d88675813","Type":"ContainerDied","Data":"c46b790ff56bfdf9891d15d30f01a18860b142ed4473a7fa4963535685282c5a"}
Aug 05 18:17:25.314835 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.314815    2562 scope.go:117] "RemoveContainer" containerID="3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"
Aug 05 18:17:25.315118 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:17:25.315095    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b\": container with ID starting with 3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b not found: ID does not exist" containerID="3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"
Aug 05 18:17:25.315245 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.315122    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b"} err="failed to get container status \"3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b\": rpc error: code = NotFound desc = could not find container \"3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b\": container with ID starting with 3de745bbf19c81402c465b1c8514edcbcba4350c1aff861fdbc6952703b9f95b not found: ID does not exist"
Aug 05 18:17:25.315245 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.315139    2562 scope.go:117] "RemoveContainer" containerID="dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"
Aug 05 18:17:25.328238 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.328220    2562 scope.go:117] "RemoveContainer" containerID="dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"
Aug 05 18:17:25.328499 ip-10-0-131-218 kubenswrapper[2562]: E0805 18:17:25.328479    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80\": container with ID starting with dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80 not found: ID does not exist" containerID="dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"
Aug 05 18:17:25.328572 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.328505    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80"} err="failed to get container status \"dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80\": rpc error: code = NotFound desc = could not find container \"dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80\": container with ID starting with dac718e477467efcc642e78718d07a0b0e9c4941742c4e71b406d603797a1e80 not found: ID does not exist"
Aug 05 18:17:25.338603 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.338581    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"]
Aug 05 18:17:25.342052 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.342030    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-6dd8dbf85-jn2vf"]
Aug 05 18:17:25.353522 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.353504    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"]
Aug 05 18:17:25.357103 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:25.357082    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-9nt54"]
Aug 05 18:17:26.079074 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.079037    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" podUID="27d889e0-9496-4c92-a859-e5237dbda52c" containerName="main" containerID="cri-o://49e5b7344b9252efe666e61987344b091bbb08389c6a7a7ae337e351350b5917" gracePeriod=45
Aug 05 18:17:26.103510 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.103485    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5933cc73-3629-44f7-a23e-ee3d88675813" path="/var/lib/kubelet/pods/5933cc73-3629-44f7-a23e-ee3d88675813/volumes"
Aug 05 18:17:26.103925 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.103910    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" path="/var/lib/kubelet/pods/a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3/volumes"
Aug 05 18:17:26.313751 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.313721    2562 generic.go:356] "Generic (PLEG): container finished" podID="27d889e0-9496-4c92-a859-e5237dbda52c" containerID="49e5b7344b9252efe666e61987344b091bbb08389c6a7a7ae337e351350b5917" exitCode=0
Aug 05 18:17:26.314103 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.313771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" event={"ID":"27d889e0-9496-4c92-a859-e5237dbda52c","Type":"ContainerDied","Data":"49e5b7344b9252efe666e61987344b091bbb08389c6a7a7ae337e351350b5917"}
Aug 05 18:17:26.335526 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.335467    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:17:26.416687 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.416657    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkr7g\" (UniqueName: \"kubernetes.io/projected/27d889e0-9496-4c92-a859-e5237dbda52c-kube-api-access-zkr7g\") pod \"27d889e0-9496-4c92-a859-e5237dbda52c\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") "
Aug 05 18:17:26.416871 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.416725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27d889e0-9496-4c92-a859-e5237dbda52c-tls-certs\") pod \"27d889e0-9496-4c92-a859-e5237dbda52c\" (UID: \"27d889e0-9496-4c92-a859-e5237dbda52c\") "
Aug 05 18:17:26.418804 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.418779    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d889e0-9496-4c92-a859-e5237dbda52c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "27d889e0-9496-4c92-a859-e5237dbda52c" (UID: "27d889e0-9496-4c92-a859-e5237dbda52c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:26.418914 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.418865    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d889e0-9496-4c92-a859-e5237dbda52c-kube-api-access-zkr7g" (OuterVolumeSpecName: "kube-api-access-zkr7g") pod "27d889e0-9496-4c92-a859-e5237dbda52c" (UID: "27d889e0-9496-4c92-a859-e5237dbda52c"). InnerVolumeSpecName "kube-api-access-zkr7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:26.517796 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.517766    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkr7g\" (UniqueName: \"kubernetes.io/projected/27d889e0-9496-4c92-a859-e5237dbda52c-kube-api-access-zkr7g\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:26.517796 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:26.517796    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27d889e0-9496-4c92-a859-e5237dbda52c-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.319945 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:27.319913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p" event={"ID":"27d889e0-9496-4c92-a859-e5237dbda52c","Type":"ContainerDied","Data":"9ea0a358537f83bbd1fd356b99d2ada77ab35ebfe2a7afcfdccb89acabbd3747"}
Aug 05 18:17:27.320379 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:27.319975    2562 scope.go:117] "RemoveContainer" containerID="49e5b7344b9252efe666e61987344b091bbb08389c6a7a7ae337e351350b5917"
Aug 05 18:17:27.320379 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:27.319928    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"
Aug 05 18:17:27.340715 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:27.340688    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"]
Aug 05 18:17:27.346016 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:27.345991    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-7976968ffdds5p"]
Aug 05 18:17:28.102285 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:28.102245    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d889e0-9496-4c92-a859-e5237dbda52c" path="/var/lib/kubelet/pods/27d889e0-9496-4c92-a859-e5237dbda52c/volumes"
Aug 05 18:17:31.132674 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:31.132597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:31.227706 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:31.227675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:36.130521 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:36.130474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:37.848488 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:37.848443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:42.296451 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:42.296412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:48.802639 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:48.802600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:54.373755 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:54.373723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:54.474142 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:54.474101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:58.070630 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:58.070562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:17:58.426140 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:17:58.426108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:04.116993 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:18:04.116957    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:19.552001 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:18:19.551959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:27.799354 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:18:27.799304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:18:27.899465 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:18:27.899440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:36.786901 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:19:36.786864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:19:42.309415 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:19:42.309381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:02.304445 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:02.304376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:05.817437 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:05.817405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:05.871389 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:05.871359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:08.463592 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:08.463565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:14.088564 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:14.088527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:14.149643 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:14.149606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:28.281825 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:28.281789    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77679b9596-fw5nw_b6e8bf52-e2e4-425a-bb29-46857e8f22f9/router/0.log"
Aug 05 18:20:29.252542 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:29.252517    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77679b9596-fw5nw_b6e8bf52-e2e4-425a-bb29-46857e8f22f9/router/0.log"
Aug 05 18:20:30.334143 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:30.334108    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cf4765b85-j5frh_daf2446e-40a8-4461-949f-290b9d37c15e/authorino/0.log"
Aug 05 18:20:30.712343 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:30.712303    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-gl8pz_6232136a-57c3-4491-bca8-80eab957578a/manager/0.log"
Aug 05 18:20:34.386680 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:34.386646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:34.519927 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:34.519892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:34.619239 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:34.619207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:37.252970 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:37.252941    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-452n4_bc5be61d-7ca6-4860-ac6d-92cf4d56ac51/global-pull-secret-syncer/0.log"
Aug 05 18:20:37.355257 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:37.355231    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log"
Aug 05 18:20:37.465277 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:37.465246    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-218.ec2.internal_0ea61056bac35de2c231d3be6b5b93be/haproxy/0.log"
Aug 05 18:20:40.917720 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:40.917689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:41.017621 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:41.017591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6grt6_0b6d9b10-5dbb-4b5e-9900-226cde1d2d48/konnectivity-agent/0.log" limit=1024
Aug 05 18:20:42.812573 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:42.812539    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7cf4765b85-j5frh_daf2446e-40a8-4461-949f-290b9d37c15e/authorino/0.log"
Aug 05 18:20:43.233949 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:43.233908    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-gl8pz_6232136a-57c3-4491-bca8-80eab957578a/manager/0.log"
Aug 05 18:20:44.427702 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.427615    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-ph4zq_4effde24-1170-4722-97ff-dda58d6a09b7/cluster-monitoring-operator/0.log"
Aug 05 18:20:44.452119 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.452092    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-w5wwp_01834f4c-8be8-4209-9d6f-949a56d6bd66/kube-state-metrics/0.log"
Aug 05 18:20:44.473286 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.473260    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-w5wwp_01834f4c-8be8-4209-9d6f-949a56d6bd66/kube-rbac-proxy-main/0.log"
Aug 05 18:20:44.497545 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.497522    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-w5wwp_01834f4c-8be8-4209-9d6f-949a56d6bd66/kube-rbac-proxy-self/0.log"
Aug 05 18:20:44.736067 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.736046    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9w6gd_a0aece35-3955-43f3-8aca-681004146d2d/node-exporter/0.log"
Aug 05 18:20:44.777374 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.777352    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9w6gd_a0aece35-3955-43f3-8aca-681004146d2d/kube-rbac-proxy/0.log"
Aug 05 18:20:44.801733 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.801712    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9w6gd_a0aece35-3955-43f3-8aca-681004146d2d/init-textfile/0.log"
Aug 05 18:20:44.977290 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.977266    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zdmxs_520f99ef-7563-4ee5-92f5-90e2b0cbdb05/kube-rbac-proxy-main/0.log"
Aug 05 18:20:44.998450 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:44.998425    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zdmxs_520f99ef-7563-4ee5-92f5-90e2b0cbdb05/kube-rbac-proxy-self/0.log"
Aug 05 18:20:45.019896 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.019876    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-zdmxs_520f99ef-7563-4ee5-92f5-90e2b0cbdb05/openshift-state-metrics/0.log"
Aug 05 18:20:45.064967 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.064948    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/prometheus/0.log"
Aug 05 18:20:45.080693 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.080675    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/config-reloader/0.log"
Aug 05 18:20:45.102546 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.102517    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/thanos-sidecar/0.log"
Aug 05 18:20:45.123879 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.123860    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/kube-rbac-proxy-web/0.log"
Aug 05 18:20:45.146955 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.146921    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/kube-rbac-proxy/0.log"
Aug 05 18:20:45.168172 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.168128    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/kube-rbac-proxy-thanos/0.log"
Aug 05 18:20:45.191718 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.191665    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71a3cd67-3ca6-4e4e-9e7f-9150fc195961/init-config-reloader/0.log"
Aug 05 18:20:45.392293 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.392264    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5774df9dc7-plqwr_82509f2b-5528-433c-a34e-63955e89c9cf/thanos-query/0.log"
Aug 05 18:20:45.411924 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.411898    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5774df9dc7-plqwr_82509f2b-5528-433c-a34e-63955e89c9cf/kube-rbac-proxy-web/0.log"
Aug 05 18:20:45.432675 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.432652    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5774df9dc7-plqwr_82509f2b-5528-433c-a34e-63955e89c9cf/kube-rbac-proxy/0.log"
Aug 05 18:20:45.453720 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.453699    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5774df9dc7-plqwr_82509f2b-5528-433c-a34e-63955e89c9cf/prom-label-proxy/0.log"
Aug 05 18:20:45.476169 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.476153    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5774df9dc7-plqwr_82509f2b-5528-433c-a34e-63955e89c9cf/kube-rbac-proxy-rules/0.log"
Aug 05 18:20:45.498933 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:45.498903    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5774df9dc7-plqwr_82509f2b-5528-433c-a34e-63955e89c9cf/kube-rbac-proxy-metrics/0.log"
Aug 05 18:20:46.670872 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:46.670840    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-2mnvv_0442c145-eda8-402a-9e11-4ee7c7f1237c/networking-console-plugin/0.log"
Aug 05 18:20:47.131839 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.131808    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/1.log"
Aug 05 18:20:47.137138 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.137115    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-5tj5j_b9544517-0371-4225-95c0-3f7c64e68bed/console-operator/2.log"
Aug 05 18:20:47.386757 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.386675    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"]
Aug 05 18:20:47.387041 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387029    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5933cc73-3629-44f7-a23e-ee3d88675813" containerName="main"
Aug 05 18:20:47.387083 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387043    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5933cc73-3629-44f7-a23e-ee3d88675813" containerName="main"
Aug 05 18:20:47.387083 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387064    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d889e0-9496-4c92-a859-e5237dbda52c" containerName="main"
Aug 05 18:20:47.387083 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387069    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d889e0-9496-4c92-a859-e5237dbda52c" containerName="main"
Aug 05 18:20:47.387179 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387083    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" containerName="main"
Aug 05 18:20:47.387179 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387090    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" containerName="main"
Aug 05 18:20:47.387179 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387143    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a495ed9b-7728-4ed1-b3c6-d3effcf0a6d3" containerName="main"
Aug 05 18:20:47.387179 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387153    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d889e0-9496-4c92-a859-e5237dbda52c" containerName="main"
Aug 05 18:20:47.387179 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.387160    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5933cc73-3629-44f7-a23e-ee3d88675813" containerName="main"
Aug 05 18:20:47.390485 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.390468    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.393000 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.392970    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgx78\"/\"openshift-service-ca.crt\""
Aug 05 18:20:47.393000 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.392995    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgx78\"/\"kube-root-ca.crt\""
Aug 05 18:20:47.393199 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.392970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zgx78\"/\"default-dockercfg-cb4lc\""
Aug 05 18:20:47.398961 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.398723    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"]
Aug 05 18:20:47.509112 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.509073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94fsm\" (UniqueName: \"kubernetes.io/projected/869b62ec-975a-4dd3-8250-6324ef234cea-kube-api-access-94fsm\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.509280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.509129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-sys\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.509280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.509169    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-proc\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.509280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.509232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-lib-modules\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.509280 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.509274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-podres\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610404 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-proc\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610594 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-lib-modules\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610594 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610453    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-podres\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610594 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610487    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-proc\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610594 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94fsm\" (UniqueName: \"kubernetes.io/projected/869b62ec-975a-4dd3-8250-6324ef234cea-kube-api-access-94fsm\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610594 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-sys\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610804 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-lib-modules\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610804 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-sys\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.610804 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.610664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/869b62ec-975a-4dd3-8250-6324ef234cea-podres\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.618126 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.618098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94fsm\" (UniqueName: \"kubernetes.io/projected/869b62ec-975a-4dd3-8250-6324ef234cea-kube-api-access-94fsm\") pod \"perf-node-gather-daemonset-qltf4\" (UID: \"869b62ec-975a-4dd3-8250-6324ef234cea\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.650543 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.650492    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-28ccc_c067cbc4-7c1e-4563-a39a-83e264d6f620/download-server/0.log"
Aug 05 18:20:47.701957 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.701939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:47.827180 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:47.827151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"]
Aug 05 18:20:47.829668 ip-10-0-131-218 kubenswrapper[2562]: W0805 18:20:47.829643    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod869b62ec_975a_4dd3_8250_6324ef234cea.slice/crio-68d603ab6a237128b19e62cb601f289bbb7486f5ed30c12ffe5fd350deed0a2e WatchSource:0}: Error finding container 68d603ab6a237128b19e62cb601f289bbb7486f5ed30c12ffe5fd350deed0a2e: Status 404 returned error can't find the container with id 68d603ab6a237128b19e62cb601f289bbb7486f5ed30c12ffe5fd350deed0a2e
Aug 05 18:20:48.192938 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:48.192912    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-ft2lq_58c51684-7e60-44ef-a39d-ac0474c834dd/volume-data-source-validator/0.log"
Aug 05 18:20:48.253109 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:48.253075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4" event={"ID":"869b62ec-975a-4dd3-8250-6324ef234cea","Type":"ContainerStarted","Data":"062952b98efef15494156d790420f1d5b93049cefe2047263b48e0a8cf66042e"}
Aug 05 18:20:48.253285 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:48.253123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4" event={"ID":"869b62ec-975a-4dd3-8250-6324ef234cea","Type":"ContainerStarted","Data":"68d603ab6a237128b19e62cb601f289bbb7486f5ed30c12ffe5fd350deed0a2e"}
Aug 05 18:20:48.253285 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:48.253189    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:48.270148 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:48.270105    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4" podStartSLOduration=1.270091538 podStartE2EDuration="1.270091538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 18:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:20:48.268446827 +0000 UTC m=+4106.801631090" watchObservedRunningTime="2026-08-05 18:20:48.270091538 +0000 UTC m=+4106.803275782"
Aug 05 18:20:49.225820 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:49.225787    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zf7p6_97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d/dns/0.log"
Aug 05 18:20:49.244031 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:49.244012    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zf7p6_97fcfdfe-9e68-4ad1-84e6-d0f5368efa5d/kube-rbac-proxy/0.log"
Aug 05 18:20:49.265151 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:49.265125    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2czsm_3cf4abb5-9d9e-4a41-9b89-d0c36b2c1826/dns-node-resolver/0.log"
Aug 05 18:20:49.846261 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:49.846234    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-69j8c_f58e0111-2bdd-4b12-8da2-d65cbf3de0b5/node-ca/0.log"
Aug 05 18:20:51.086698 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:51.086670    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77679b9596-fw5nw_b6e8bf52-e2e4-425a-bb29-46857e8f22f9/router/0.log"
Aug 05 18:20:51.558451 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:51.558417    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dgm48_290d541c-fa30-4f67-b8ba-0360072ef692/serve-healthcheck-canary/0.log"
Aug 05 18:20:52.026277 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:52.026246    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-97hsq_8f9aa898-2769-4d57-9eac-c585ad07a01a/insights-operator/0.log"
Aug 05 18:20:52.028245 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:52.028214    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-97hsq_8f9aa898-2769-4d57-9eac-c585ad07a01a/insights-operator/1.log"
Aug 05 18:20:52.048847 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:52.048803    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4wxcg_a49fae96-7a78-4f89-b998-976654a7ae49/kube-rbac-proxy/0.log"
Aug 05 18:20:52.070513 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:52.070498    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4wxcg_a49fae96-7a78-4f89-b998-976654a7ae49/exporter/0.log"
Aug 05 18:20:52.092183 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:52.092163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4wxcg_a49fae96-7a78-4f89-b998-976654a7ae49/extractor/0.log"
Aug 05 18:20:54.267723 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:54.267687    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-qltf4"
Aug 05 18:20:55.274462 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:55.274439    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77c595c7bb-96fq9_84950fef-5a5e-4abe-9ef5-2fbb89a4f582/manager/0.log"
Aug 05 18:20:55.296233 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:55.296207    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nk6rm_cb58a5c9-6746-416a-985b-1100145b716a/openshift-lws-operator/0.log"
Aug 05 18:20:57.432727 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:57.432701    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77679b9596-fw5nw_b6e8bf52-e2e4-425a-bb29-46857e8f22f9/router/0.log"
Aug 05 18:20:58.851618 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:20:58.851589    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77679b9596-fw5nw_b6e8bf52-e2e4-425a-bb29-46857e8f22f9/router/0.log"
Aug 05 18:21:03.224831 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:21:03.224787    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-q5wr7_df0256bf-e00b-43ec-96ef-ad9e6f2717c9/kube-storage-version-migrator-operator/1.log"
Aug 05 18:21:03.225633 ip-10-0-131-218 kubenswrapper[2562]: I0805 18:21:03.225613    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-q5wr7_df0256bf-e00b-43ec-96ef-ad9e6f2717c9/kube-storage-version-migrator-operator/0.log"