Jul 27 17:26:44.383143 ip-10-0-135-40 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 27 17:26:44.383152 ip-10-0-135-40 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 27 17:26:44.383159 ip-10-0-135-40 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 27 17:26:44.383433 ip-10-0-135-40 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 27 17:26:54.590267 ip-10-0-135-40 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 27 17:26:54.590283 ip-10-0-135-40 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 748cc82d2c264219bc86f43799604ed1 --
Jul 27 17:28:51.810718 ip-10-0-135-40 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 17:28:52.195527 ip-10-0-135-40 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:28:52.195527 ip-10-0-135-40 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 17:28:52.195527 ip-10-0-135-40 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:28:52.195527 ip-10-0-135-40 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 17:28:52.195527 ip-10-0-135-40 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 17:28:52.196213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.195579    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 17:28:52.202624 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202605    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:28:52.202624 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202620    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:28:52.202624 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202623    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:28:52.202624 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202627    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202630    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202633    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202636    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202641    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202644    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202647    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202649    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202652    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202654    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202657    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202660    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202662    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202665    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202673    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202675    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202678    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202681    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202684    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202687    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:28:52.202740 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202689    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202694    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202697    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202700    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202703    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202706    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202708    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202711    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202714    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202717    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202719    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202722    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202725    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202728    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202731    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202733    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202736    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202739    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202742    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202744    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:28:52.203571 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202747    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202750    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202752    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202755    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202758    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202760    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202764    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202766    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202769    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202772    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202774    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202777    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202780    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202782    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202787    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202791    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202794    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202796    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202799    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202802    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:28:52.204282 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202805    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202808    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202811    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202814    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202817    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202820    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202823    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202826    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202829    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202832    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202835    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202837    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202840    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202845    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202849    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202852    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202855    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202858    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202861    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:28:52.204788 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202864    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202867    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202869    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202872    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202875    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202877    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202880    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202883    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202885    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202888    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202891    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202893    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202896    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202899    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202902    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202905    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:28:52.205340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.202908    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:28:52.206071 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206054    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:28:52.206071 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206071    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206077    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206081    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206085    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206088    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206091    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206094    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206097    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206099    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206102    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206105    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206107    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206110    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206113    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206117    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206121    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206124    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206127    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:28:52.206126 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206130    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206141    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206144    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206147    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206149    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206153    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206156    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206160    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206163    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206166    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206169    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206173    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206176    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206178    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206181    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206184    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206187    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206190    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206192    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:28:52.206627 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206194    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206197    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206199    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206202    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206204    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206207    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206210    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206212    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206214    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206217    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206220    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206223    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206225    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206228    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206230    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206234    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206237    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206239    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206242    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206245    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:28:52.207089 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206247    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206250    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206252    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206255    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206257    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206260    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206262    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206265    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206267    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206269    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206272    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206274    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206277    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206279    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206282    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206284    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206287    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206289    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206291    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206294    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:28:52.207612 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206296    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206299    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206301    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206304    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206306    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206310    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206312    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206316    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206318    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206321    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206324    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206326    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206329    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206331    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206334    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206336    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206339    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206341    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206344    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:28:52.208096 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206346    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206349    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206759    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206765    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206769    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206772    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206775    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206778    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206781    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206783    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206786    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206789    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206791    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206794    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206797    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206799    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206802    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206804    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206808    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206811    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:28:52.208575 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206814    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206817    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206820    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206823    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206825    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206828    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206830    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206833    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206835    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206838    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206840    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206843    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206845    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206848    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206851    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206853    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206856    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206859    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206861    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206864    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:28:52.209055 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206867    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206869    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206872    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206874    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206877    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206879    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206882    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206885    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206888    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206890    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206893    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206896    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206900    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206902    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206904    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206907    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206912    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206915    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206918    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:28:52.209541 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206921    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206924    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206927    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206929    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206932    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206934    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206937    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206939    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206942    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206945    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206947    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206950    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206952    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206955    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206957    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206960    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206962    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206965    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206968    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206970    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:28:52.209999 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206973    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206976    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206978    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206981    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206985    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206989    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206992    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.206996    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207000    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207003    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207005    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207008    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207013    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207016    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207018    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207021    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207023    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207026    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207029    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:28:52.210484 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207031    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207034    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207037    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207111    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207121    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207140    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207146    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207152    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207157    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207162    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207166    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207169    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207172    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207175    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207179    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207182    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207185    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207188    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207191    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207194    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207197    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207202    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207205    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207208    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 17:28:52.210943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207211    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207215    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207219    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207222    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207225    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207228    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207231    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207234    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207237    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207240    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207243    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207247    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207250    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207253    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207256    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207259    2577 flags.go:64] FLAG: --enable-server="true"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207262    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207266    2577 flags.go:64] FLAG: --event-burst="100"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207269    2577 flags.go:64] FLAG: --event-qps="50"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207272    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207275    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207277    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207282    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207285    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207288    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 17:28:52.211538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207290    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207293    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207296    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207300    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207303    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207306    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207309    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207312    2577 flags.go:64] FLAG: --feature-gates=""
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207316    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207319    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207322    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207325    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207328    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207331    2577 flags.go:64] FLAG: --help="false"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207334    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207337    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207340    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207343    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207346    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207352    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207355    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207358    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207361    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 17:28:52.212171 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207364    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207367    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207370    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207373    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207375    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207378    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207381    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207384    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207387    2577 flags.go:64] FLAG: --lock-file=""
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207389    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207392    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207395    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207410    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207414    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207417    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207419    2577 flags.go:64] FLAG: --logging-format="text"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207422    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207437    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207440    2577 flags.go:64] FLAG: --manifest-url=""
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207443    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207447    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207451    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207455    2577 flags.go:64] FLAG: --max-pods="110"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207458    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207461    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 17:28:52.212780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207464    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207467    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207470    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207474    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207477    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207484    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207487    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207490    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207493    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207496    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207501    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207504    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207507    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207510    2577 flags.go:64] FLAG: --port="10250"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207513    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207516    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02bfb6567cbcbb425"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207519    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207522    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207525    2577 flags.go:64] FLAG: --register-node="true"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207528    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207532    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207535    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207538    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207541    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207544    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 17:28:52.213378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207548    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207551    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207554    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207556    2577 flags.go:64] FLAG: --runonce="false"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207560    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207563    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207566    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207569    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207572    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207575    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207578    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207582    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207585    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207588    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207592    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207594    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207597    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207600    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207603    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207608    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207611    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207614    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207618    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207621    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207624    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 17:28:52.214038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207627    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207630    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207634    2577 flags.go:64] FLAG: --v="2"
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207638    2577 flags.go:64] FLAG: --version="false"
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207642    2577 flags.go:64] FLAG: --vmodule=""
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207646    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.207651    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207778    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207782    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207785    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207788    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207790    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207793    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207796    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207799    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207801    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207804    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207807    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207813    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207816    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207818    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:28:52.214675 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207821    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207824    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207826    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207829    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207832    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207834    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207837    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207839    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207842    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207845    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207847    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207850    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207853    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207856    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207858    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207861    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207864    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207870    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207873    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207876    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:28:52.215179 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207878    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207881    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207885    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207889    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207892    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207895    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207898    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207901    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207903    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207907    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207910    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207913    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207916    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207919    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207922    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207925    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207927    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207930    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207932    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:28:52.215724 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207935    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207937    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207940    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207943    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207945    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207948    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207950    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207953    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207955    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207958    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207962    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207965    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207968    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207970    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207973    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207975    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207978    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207980    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207983    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:28:52.216199 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207985    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207988    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207991    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.207997    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208000    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208002    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208005    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208008    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208011    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208014    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208017    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208020    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208023    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208025    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208028    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208030    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208032    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208035    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208037    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:28:52.216684 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208040    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208042    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208045    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208047    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208051    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208054    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208057    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.208060    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.208729    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.215164    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.215180    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.215193    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.215200    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215255    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215260    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:28:52.217152 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215263    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215267    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215270    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215274    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215278    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215281    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215284    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215286    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215289    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215292    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215295    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215298    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215300    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215303    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215306    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215309    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215311    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215314    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215317    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:28:52.217549 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215319    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215321    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215324    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215326    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215329    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215332    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215334    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215337    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215341    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215344    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215347    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215349    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215352    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215355    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215357    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215360    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215362    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215365    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215367    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:28:52.218003 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215370    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215374    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215378    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215381    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215384    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215387    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215389    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215392    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215394    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215405    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215409    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215411    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215414    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215417    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215419    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215422    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215437    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215442    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215446    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215450    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:28:52.218499 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215453    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215456    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215460    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215463    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215466    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215468    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215471    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215473    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215476    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215479    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215481    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215483    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215486    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215488    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215491    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215493    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215497    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215499    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215501    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215504    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:28:52.219008 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215506    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215509    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215512    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215514    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215517    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215520    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215524    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215527    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215530    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215533    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215536    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215538    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215541    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215544    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215547    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215550    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215553    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215556    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:28:52.219598 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215558    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.215563    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215678    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215683    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215687    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215690    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215694    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215697    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215699    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215702    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215706    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215708    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215711    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215713    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 17:28:52.220059 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215716    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215718    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215721    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215724    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215726    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215729    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215732    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215735    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215738    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215740    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215743    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215746    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215749    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215752    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215754    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215757    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215760    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215763    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215765    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215768    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 17:28:52.220420 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215770    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215773    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215775    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215778    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215780    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215783    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215785    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215788    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215792    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215795    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215798    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215800    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215803    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215806    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215808    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215810    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215813    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215815    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215818    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 17:28:52.221340 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215821    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215823    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215825    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215829    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215831    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215834    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215836    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215839    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215842    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215845    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215848    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215851    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215853    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215855    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215858    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215860    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215863    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215865    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215868    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215870    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 17:28:52.221948 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215873    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215875    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215878    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215880    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215882    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215885    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215888    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215892    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215895    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215897    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215900    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215902    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215904    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215907    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215910    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215912    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215915    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215917    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215920    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 17:28:52.222569 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215923    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215925    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215928    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215930    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215933    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215936    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215938    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215940    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:52.215943    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.215948    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.216049    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.218115    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.219335    2577 server.go:1019] "Starting client certificate rotation"
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.219454    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:28:52.223160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.219494    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 17:28:52.223548 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.221671    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:28:52.224463 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.224449    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 17:28:52.237952 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.237935    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 27 17:28:52.240834 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.240816    2577 log.go:25] "Validated CRI v1 image API"
Jul 27 17:28:52.241895 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.241877    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 17:28:52.244203 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.244183    2577 fs.go:135] Filesystem UUIDs: map[6721967f-4f63-4f56-8bfd-c5d04ecaccf4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 863abd00-5cc8-481d-be34-ec2b2967cc08:/dev/nvme0n1p3]
Jul 27 17:28:52.244267 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.244202    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 17:28:52.248661 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.248556    2577 manager.go:216] Machine: {Timestamp:2026-07-27 17:28:52.248058344 +0000 UTC m=+0.326490757 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099491 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23ab64993b7d51ff99e8ba101746c0 SystemUUID:ec23ab64-993b-7d51-ff99-e8ba101746c0 BootID:748cc82d-2c26-4219-bc86-f43799604ed1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:9d:ff:c6:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:9d:ff:c6:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:bd:83:3c:09:59 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 17:28:52.248661 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.248656    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 17:28:52.248786 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.248754    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 17:28:52.248880 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.248865    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:28:52.250378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.250357    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 17:28:52.250547 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.250380    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 17:28:52.251928 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.251917    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 17:28:52.251928 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.251929    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 17:28:52.252011 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.251936    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:28:52.252645 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.252635    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 17:28:52.253990 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.253980    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 17:28:52.255938 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.255927    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:28:52.256046 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256036    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 17:28:52.256077 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256064    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 17:28:52.256077 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256074    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 17:28:52.256133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256089    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 17:28:52.256133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256098    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 27 17:28:52.256133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256106    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 17:28:52.256501 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256485    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:28:52.256544 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.256520    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 17:28:52.258835 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.258816    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 17:28:52.260625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.260612    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 17:28:52.260679 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.260642    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 17:28:52.262267 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262251    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262277    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262289    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262298    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262307    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262313    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262321    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 17:28:52.262324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262327    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 17:28:52.262625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262335    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 17:28:52.262625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262341    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 17:28:52.262625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262352    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 17:28:52.262625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262360    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 17:28:52.262625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262384    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 17:28:52.262625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.262391    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 17:28:52.266194 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.266174    2577 server.go:1268] "Started kubelet"
Jul 27 17:28:52.266950 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.266806    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 17:28:52.267319 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.267299    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 17:28:52.267348 ip-10-0-135-40 systemd[1]: Started Kubernetes Kubelet.
Jul 27 17:28:52.267530 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.267400    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 17:28:52.267795 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.267645    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 17:28:52.267795 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.267746    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 17:28:52.267931 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.267835    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 17:28:52.270001 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.269979    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 17:28:52.270618 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.270604    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 17:28:52.273684 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.273665    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7j5h"
Jul 27 17:28:52.276440 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.274759    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18c635affc391404  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-27 17:28:52.266128388 +0000 UTC m=+0.344560815,LastTimestamp:2026-07-27 17:28:52.266128388 +0000 UTC m=+0.344560815,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}"
Jul 27 17:28:52.276561 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.276545    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 17:28:52.277202 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277183    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 17:28:52.277202 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277200    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 17:28:52.277406 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.277387    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.277875 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277854    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 17:28:52.277875 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277855    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 17:28:52.278017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277887    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 17:28:52.278017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277963    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 17:28:52.278017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277974    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 17:28:52.278017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.277988    2577 factory.go:55] Registering systemd factory
Jul 27 17:28:52.278198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.278023    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 27 17:28:52.278304 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.278290    2577 factory.go:153] Registering CRI-O factory
Jul 27 17:28:52.278358 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.278309    2577 factory.go:223] Registration of the crio container factory successfully
Jul 27 17:28:52.278406 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.278363    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 17:28:52.278406 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.278388    2577 factory.go:103] Registering Raw factory
Jul 27 17:28:52.278521 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.278481    2577 manager.go:1196] Started watching for new ooms in manager
Jul 27 17:28:52.279212 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.279193    2577 manager.go:349] Starting recovery of all containers
Jul 27 17:28:52.282324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.282303    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7j5h"
Jul 27 17:28:52.284865 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.284841    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 17:28:52.285139 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.285097    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 17:28:52.289694 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.289445    2577 manager.go:354] Recovery completed
Jul 27 17:28:52.291612 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.291594    2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 27 17:28:52.295063 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.295050    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.296796 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.296779    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.296864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.296819    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.296864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.296830    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.297412 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.297397    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 17:28:52.297412 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.297411    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 17:28:52.297537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.297427    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 17:28:52.297590 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.297578    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.298261 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.298245    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.298335 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.298277    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.298335 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.298291    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.299349 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.299329    2577 policy_none.go:49] "None policy: Start"
Jul 27 17:28:52.299415 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.299353    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 17:28:52.299415 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.299368    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 17:28:52.299415 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.299317    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18c635affe0d094b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-27 17:28:52.296796491 +0000 UTC m=+0.375228910,LastTimestamp:2026-07-27 17:28:52.296796491 +0000 UTC m=+0.375228910,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}"
Jul 27 17:28:52.300572 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.300559    2577 policy_none.go:47] "Start"
Jul 27 17:28:52.335983 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.335967    2577 manager.go:337] "Starting Device Plugin manager"
Jul 27 17:28:52.336077 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.336006    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 17:28:52.336077 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.336019    2577 server.go:87] "Starting device plugin registration server"
Jul 27 17:28:52.336274 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.336260    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 17:28:52.336332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.336275    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 17:28:52.336396 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.336377    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 17:28:52.336488 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.336470    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 17:28:52.336488 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.336486    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 17:28:52.336982 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.336965    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 17:28:52.337062 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.337011    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.438347 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.438319    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.439630 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.439473    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.439630 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.439504    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.439630 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.439519    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.439630 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.439561    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.450101 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.450047    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.450101 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.450076    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-40.ec2.internal\": node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.470008 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.469983    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.475439 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.475414    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 17:28:52.476645 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.476631    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 17:28:52.476707 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.476668    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 17:28:52.476707 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.476691    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 17:28:52.476809 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.476723    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 17:28:52.479796 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.479779    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:52.571025 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.570990    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.577698 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.577673    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"]
Jul 27 17:28:52.577758 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.577741    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.578245 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.578225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55803ecb689b631e773ecebce7207b60-config\") pod \"kube-apiserver-proxy-ip-10-0-135-40.ec2.internal\" (UID: \"55803ecb689b631e773ecebce7207b60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.579408 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.579391    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.579497 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.579429    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.579497 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.579464    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.581279 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.581266    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.581391 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.581377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.581452 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.581405    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.582043 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.582028    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.582112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.582030    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.582112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.582052    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.582112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.582062    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.582112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.582065    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.582112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.582076    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.583379 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.583363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.583433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.583394    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 17:28:52.584111 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.584094    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 17:28:52.584175 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.584121    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 17:28:52.584175 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.584136    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID"
Jul 27 17:28:52.601080 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.601056    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-40.ec2.internal\" not found" node="ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.605898 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.605881    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-40.ec2.internal\" not found" node="ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.671799 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.671769    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.679000 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.678977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55803ecb689b631e773ecebce7207b60-config\") pod \"kube-apiserver-proxy-ip-10-0-135-40.ec2.internal\" (UID: \"55803ecb689b631e773ecebce7207b60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.679092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.679038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55803ecb689b631e773ecebce7207b60-config\") pod \"kube-apiserver-proxy-ip-10-0-135-40.ec2.internal\" (UID: \"55803ecb689b631e773ecebce7207b60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.772095 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.772055    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.779959 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.779935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f24c4d72d8c15a7ea1fde556db108ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"1f24c4d72d8c15a7ea1fde556db108ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.780012 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.779969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f24c4d72d8c15a7ea1fde556db108ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"1f24c4d72d8c15a7ea1fde556db108ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.879773 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.879730    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:52.880950 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.880931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f24c4d72d8c15a7ea1fde556db108ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"1f24c4d72d8c15a7ea1fde556db108ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.881007 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.880961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f24c4d72d8c15a7ea1fde556db108ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"1f24c4d72d8c15a7ea1fde556db108ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.881046 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.881004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f24c4d72d8c15a7ea1fde556db108ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"1f24c4d72d8c15a7ea1fde556db108ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.881046 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.881036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f24c4d72d8c15a7ea1fde556db108ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"1f24c4d72d8c15a7ea1fde556db108ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.904489 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.904457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.909958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:52.909941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:52.980885 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:52.980846    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:53.082139 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.082069    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:53.121498 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.121470    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:53.182282 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.182252    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found"
Jul 27 17:28:53.182611 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.182593    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:53.220015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.219996    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 17:28:53.220620 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.220107    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:28:53.220620 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.220162    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:28:53.220620 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.220162    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 17:28:53.258204 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.258173    2577 apiserver.go:52] "Watching apiserver"
Jul 27 17:28:53.277862 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.277834    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 17:28:53.277971 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.277873    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:53.278016 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.277875    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 17:28:53.280244 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.279786    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-6lb2p","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn","openshift-cluster-node-tuning-operator/tuned-7hkzw","openshift-image-registry/node-ca-72mk7","openshift-multus/multus-additional-cni-plugins-bpmlj","openshift-multus/multus-gdzng","openshift-ovn-kubernetes/ovnkube-node-nld5j","kube-system/global-pull-secret-syncer-qxkrc","openshift-dns/node-resolver-w84kh","openshift-multus/network-metrics-daemon-29lnk","openshift-network-diagnostics/network-check-target-fncdg","openshift-network-operator/iptables-alerter-lw92s"]
Jul 27 17:28:53.281639 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.281617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.283220 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.283195    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.284285 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.284253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 17:28:53.284393 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.284361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.284467 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.284420    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fnh6h\""
Jul 27 17:28:53.284535 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.284491    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 17:28:53.285840 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.285821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.285974 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.285952    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.286068 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.285921    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tcmmz\""
Jul 27 17:28:53.286068 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.285959    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.286182 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.285944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 17:28:53.287156 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.287126    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.287156 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.287148    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m94gx\""
Jul 27 17:28:53.287303 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.287151    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.287303 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.287254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.287674 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.287656    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:28:53.287768 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.287726    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"
Jul 27 17:28:53.288554 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.288537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.288708 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.288690    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.288822 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.288804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-25q8b\""
Jul 27 17:28:53.288956 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.288941    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 17:28:53.289041 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.289004    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 17:23:52 +0000 UTC" deadline="2028-03-13 15:02:53.419325494 +0000 UTC"
Jul 27 17:28:53.289087 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.289041    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14277h34m0.130287162s"
Jul 27 17:28:53.289288 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.289272    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.289733 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.289714    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.289820 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.289768    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.289862 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.289835    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 17:28:53.290069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.290056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.290354 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.290332    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 17:28:53.290453 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.290370    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 17:28:53.290453 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.290383    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 17:28:53.290756 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.290741    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rd5k7\""
Jul 27 17:28:53.291399 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.291383    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hpmw9\""
Jul 27 17:28:53.291481 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.291470    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.291614 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.291515    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:28:53.292273 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.292259    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 17:28:53.292621 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.292606    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 17:28:53.292769 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.292754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.293660 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.293639    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 17:28:53.293771 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.293688    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 17:28:53.293771 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.293694    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.293888 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.293838    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2g5mv\""
Jul 27 17:28:53.294145 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.294118    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 17:28:53.294145 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.294139    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.294297 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.294280    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.294350 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.294328    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:28:53.295152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.295136    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.295255 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.295236    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.295320 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.295275    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fljwx\""
Jul 27 17:28:53.295961 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.295942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:53.296050 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.296004    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:28:53.297468 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.297452    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"]
Jul 27 17:28:53.297549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.297537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.300407 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.300387    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 17:28:53.300520 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.300391    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:28:53.300575 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.300538    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c7m4d\""
Jul 27 17:28:53.300628 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.300601    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 17:28:53.301882 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.301868    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 17:28:53.302325 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.302312    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal"]
Jul 27 17:28:53.313124 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.313106    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rr5qx"
Jul 27 17:28:53.323351 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.323333    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rr5qx"
Jul 27 17:28:53.378688 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.378667    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 17:28:53.382775 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-etc-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.382775 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-cni-netd\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.382775 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-env-overrides\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.382937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c9cb2437-7ae9-44d4-8f5f-927e38a41769-dbus\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.382937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.382937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-conf-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.382937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e52ffd-2d0c-4934-9492-07647e3e1a50-tmp\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.383072 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-systemd-units\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383072 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.382970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-slash\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383072 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5360283-57fc-49d4-87bf-9322bf5e5665-ovn-node-metrics-cert\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383072 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-cni-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383072 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-multus-certs\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383263 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgk74\" (UniqueName: \"kubernetes.io/projected/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-kube-api-access-xgk74\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.383263 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgz7x\" (UniqueName: \"kubernetes.io/projected/e48eca8f-20da-4ade-9817-c88f1a443e89-kube-api-access-mgz7x\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.383263 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-modprobe-d\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.383263 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-system-cni-dir\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.383263 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-run-netns\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-k8s-cni-cncf-io\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-etc-kubernetes\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-systemd\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-var-lib-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-cni-bin\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-ovnkube-script-lib\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-cnibin\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-os-release\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383598    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48c0dcb3-16e3-46a1-ba47-a91186ce974b-cni-binary-copy\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/72c414a3-2b52-4f98-965c-dad0bc417b76-iptables-alerter-script\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.383709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72c414a3-2b52-4f98-965c-dad0bc417b76-host-slash\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-etc-selinux\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-var-lib-kubelet\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cnibin\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-lib-modules\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4wjp\" (UniqueName: \"kubernetes.io/projected/75d73dec-5e9b-48a8-882b-4f67668fb911-kube-api-access-m4wjp\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-log-socket\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-ovnkube-config\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-hostroot\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-host\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.383981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.383973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75d73dec-5e9b-48a8-882b-4f67668fb911-host\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bc14a53-bf9c-4c74-8b11-2953f0979658-konnectivity-ca\") pod \"konnectivity-agent-6lb2p\" (UID: \"8bc14a53-bf9c-4c74-8b11-2953f0979658\") " pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-socket-dir-parent\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kqr8\" (UniqueName: \"kubernetes.io/projected/48c0dcb3-16e3-46a1-ba47-a91186ce974b-kube-api-access-9kqr8\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnd9c\" (UniqueName: \"kubernetes.io/projected/c7d5444a-98d9-4fe7-8017-a35216c24332-kube-api-access-rnd9c\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-hosts-file\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-sys-fs\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-tuned\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgt2v\" (UniqueName: \"kubernetes.io/projected/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-kube-api-access-dgt2v\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-ovn\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxsg\" (UniqueName: \"kubernetes.io/projected/c5360283-57fc-49d4-87bf-9322bf5e5665-kube-api-access-5cxsg\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-daemon-config\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-kubernetes\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysctl-conf\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-sys\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-cni-multus\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-kubelet\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bg5m\" (UniqueName: \"kubernetes.io/projected/72c414a3-2b52-4f98-965c-dad0bc417b76-kube-api-access-4bg5m\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysctl-d\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsr27\" (UniqueName: \"kubernetes.io/projected/d2e52ffd-2d0c-4934-9492-07647e3e1a50-kube-api-access-tsr27\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c9cb2437-7ae9-44d4-8f5f-927e38a41769-kubelet-config\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-netns\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-kubelet\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-device-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-systemd\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-node-log\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.384782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-run-ovn-kubernetes\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-registration-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysconfig\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-os-release\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-system-cni-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-cni-bin\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-socket-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bc14a53-bf9c-4c74-8b11-2953f0979658-agent-certs\") pod \"konnectivity-agent-6lb2p\" (UID: \"8bc14a53-bf9c-4c74-8b11-2953f0979658\") " pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-tmp-dir\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-run\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.385302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.384929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75d73dec-5e9b-48a8-882b-4f67668fb911-serviceca\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.485182 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-k8s-cni-cncf-io\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485182 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-etc-kubernetes\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-systemd\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-etc-kubernetes\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-var-lib-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-cni-bin\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-k8s-cni-cncf-io\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-ovnkube-script-lib\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.485435 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-systemd\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-cnibin\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-cni-bin\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-cnibin\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-var-lib-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-os-release\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48c0dcb3-16e3-46a1-ba47-a91186ce974b-cni-binary-copy\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-os-release\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/72c414a3-2b52-4f98-965c-dad0bc417b76-iptables-alerter-script\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72c414a3-2b52-4f98-965c-dad0bc417b76-host-slash\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-etc-selinux\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-var-lib-kubelet\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cnibin\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-lib-modules\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wjp\" (UniqueName: \"kubernetes.io/projected/75d73dec-5e9b-48a8-882b-4f67668fb911-kube-api-access-m4wjp\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-log-socket\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-ovnkube-config\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.485899 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-hostroot\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-host\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75d73dec-5e9b-48a8-882b-4f67668fb911-host\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bc14a53-bf9c-4c74-8b11-2953f0979658-konnectivity-ca\") pod \"konnectivity-agent-6lb2p\" (UID: \"8bc14a53-bf9c-4c74-8b11-2953f0979658\") " pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-socket-dir-parent\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.485996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kqr8\" (UniqueName: \"kubernetes.io/projected/48c0dcb3-16e3-46a1-ba47-a91186ce974b-kube-api-access-9kqr8\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnd9c\" (UniqueName: \"kubernetes.io/projected/c7d5444a-98d9-4fe7-8017-a35216c24332-kube-api-access-rnd9c\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-hosts-file\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-sys-fs\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-tuned\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgt2v\" (UniqueName: \"kubernetes.io/projected/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-kube-api-access-dgt2v\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-ovn\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-ovnkube-script-lib\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxsg\" (UniqueName: \"kubernetes.io/projected/c5360283-57fc-49d4-87bf-9322bf5e5665-kube-api-access-5cxsg\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-daemon-config\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.486549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-kubernetes\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysctl-conf\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-sys\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-cni-multus\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-kubelet\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bg5m\" (UniqueName: \"kubernetes.io/projected/72c414a3-2b52-4f98-965c-dad0bc417b76-kube-api-access-4bg5m\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysctl-d\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsr27\" (UniqueName: \"kubernetes.io/projected/d2e52ffd-2d0c-4934-9492-07647e3e1a50-kube-api-access-tsr27\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-cni-multus\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48c0dcb3-16e3-46a1-ba47-a91186ce974b-cni-binary-copy\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c9cb2437-7ae9-44d4-8f5f-927e38a41769-kubelet-config\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-netns\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-kubernetes\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-kubelet\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487069 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysctl-d\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-device-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486643    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cnibin\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-systemd\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-var-lib-kubelet\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-daemon-config\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-node-log\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72c414a3-2b52-4f98-965c-dad0bc417b76-host-slash\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-etc-selinux\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-node-log\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-run-ovn-kubernetes\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c9cb2437-7ae9-44d4-8f5f-927e38a41769-kubelet-config\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-sys-fs\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-log-socket\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-sys\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/72c414a3-2b52-4f98-965c-dad0bc417b76-iptables-alerter-script\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-hostroot\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-host\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.487776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75d73dec-5e9b-48a8-882b-4f67668fb911-host\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.486977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-run-ovn\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-netns\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-registration-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-lib-modules\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysconfig\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-os-release\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-system-cni-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-cni-bin\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-socket-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487267    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bc14a53-bf9c-4c74-8b11-2953f0979658-agent-certs\") pod \"konnectivity-agent-6lb2p\" (UID: \"8bc14a53-bf9c-4c74-8b11-2953f0979658\") " pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487279    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-tmp-dir\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-kubelet\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-hosts-file\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.488715 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.487379    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-socket-dir-parent\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.487437    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:53.987417746 +0000 UTC m=+2.065850153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-os-release\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-registration-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8bc14a53-bf9c-4c74-8b11-2953f0979658-konnectivity-ca\") pod \"konnectivity-agent-6lb2p\" (UID: \"8bc14a53-bf9c-4c74-8b11-2953f0979658\") " pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysconfig\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-cni-bin\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-sysctl-conf\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487928    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-systemd\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-system-cni-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-ovnkube-config\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-device-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-var-lib-kubelet\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.487271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-cni-binary-copy\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-run-ovn-kubernetes\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.489671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e48eca8f-20da-4ade-9817-c88f1a443e89-socket-dir\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-run\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75d73dec-5e9b-48a8-882b-4f67668fb911-serviceca\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-etc-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-cni-netd\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-run\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-env-overrides\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c9cb2437-7ae9-44d4-8f5f-927e38a41769-dbus\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-etc-openvswitch\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-tmp-dir\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-conf-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e52ffd-2d0c-4934-9492-07647e3e1a50-tmp\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-systemd-units\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-slash\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5360283-57fc-49d4-87bf-9322bf5e5665-ovn-node-metrics-cert\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-cni-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-multus-certs\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.490596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgk74\" (UniqueName: \"kubernetes.io/projected/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-kube-api-access-xgk74\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgz7x\" (UniqueName: \"kubernetes.io/projected/e48eca8f-20da-4ade-9817-c88f1a443e89-kube-api-access-mgz7x\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-modprobe-d\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-cni-netd\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-system-cni-dir\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-slash\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-run-netns\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-host-run-netns\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75d73dec-5e9b-48a8-882b-4f67668fb911-serviceca\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c9cb2437-7ae9-44d4-8f5f-927e38a41769-dbus\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5360283-57fc-49d4-87bf-9322bf5e5665-env-overrides\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488786    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-host-run-multus-certs\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-conf-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.488824    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.488880    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:53.988859656 +0000 UTC m=+2.067292065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.488946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48c0dcb3-16e3-46a1-ba47-a91186ce974b-multus-cni-dir\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.491433 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.489143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-modprobe-d\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.492026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.489179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-system-cni-dir\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.492026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.489190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5360283-57fc-49d4-87bf-9322bf5e5665-systemd-units\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.492026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.490652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2e52ffd-2d0c-4934-9492-07647e3e1a50-etc-tuned\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.492026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.490927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8bc14a53-bf9c-4c74-8b11-2953f0979658-agent-certs\") pod \"konnectivity-agent-6lb2p\" (UID: \"8bc14a53-bf9c-4c74-8b11-2953f0979658\") " pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.492026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.491226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5360283-57fc-49d4-87bf-9322bf5e5665-ovn-node-metrics-cert\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.492026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.491410    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2e52ffd-2d0c-4934-9492-07647e3e1a50-tmp\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.494928 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.494906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsr27\" (UniqueName: \"kubernetes.io/projected/d2e52ffd-2d0c-4934-9492-07647e3e1a50-kube-api-access-tsr27\") pod \"tuned-7hkzw\" (UID: \"d2e52ffd-2d0c-4934-9492-07647e3e1a50\") " pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.497674 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.497657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4wjp\" (UniqueName: \"kubernetes.io/projected/75d73dec-5e9b-48a8-882b-4f67668fb911-kube-api-access-m4wjp\") pod \"node-ca-72mk7\" (UID: \"75d73dec-5e9b-48a8-882b-4f67668fb911\") " pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.499749 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.499728    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:28:53.499749 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.499752    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:28:53.499895 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.499769    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:53.499895 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.499845    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:53.999826847 +0000 UTC m=+2.078259273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:53.502467 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.502420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgt2v\" (UniqueName: \"kubernetes.io/projected/9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8-kube-api-access-dgt2v\") pod \"multus-additional-cni-plugins-bpmlj\" (UID: \"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8\") " pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.502583 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.502483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxsg\" (UniqueName: \"kubernetes.io/projected/c5360283-57fc-49d4-87bf-9322bf5e5665-kube-api-access-5cxsg\") pod \"ovnkube-node-nld5j\" (UID: \"c5360283-57fc-49d4-87bf-9322bf5e5665\") " pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.502781 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.502758    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:53.503316 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.503294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bg5m\" (UniqueName: \"kubernetes.io/projected/72c414a3-2b52-4f98-965c-dad0bc417b76-kube-api-access-4bg5m\") pod \"iptables-alerter-lw92s\" (UID: \"72c414a3-2b52-4f98-965c-dad0bc417b76\") " pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.504003 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.503979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnd9c\" (UniqueName: \"kubernetes.io/projected/c7d5444a-98d9-4fe7-8017-a35216c24332-kube-api-access-rnd9c\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.504680 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.504652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgk74\" (UniqueName: \"kubernetes.io/projected/1cfaef06-f66e-4e0d-9cae-01a4f177dd39-kube-api-access-xgk74\") pod \"node-resolver-w84kh\" (UID: \"1cfaef06-f66e-4e0d-9cae-01a4f177dd39\") " pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.504907 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.504892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgz7x\" (UniqueName: \"kubernetes.io/projected/e48eca8f-20da-4ade-9817-c88f1a443e89-kube-api-access-mgz7x\") pod \"aws-ebs-csi-driver-node-m82gn\" (UID: \"e48eca8f-20da-4ade-9817-c88f1a443e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.505232 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.505217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kqr8\" (UniqueName: \"kubernetes.io/projected/48c0dcb3-16e3-46a1-ba47-a91186ce974b-kube-api-access-9kqr8\") pod \"multus-gdzng\" (UID: \"48c0dcb3-16e3-46a1-ba47-a91186ce974b\") " pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.548363 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.548335    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f24c4d72d8c15a7ea1fde556db108ca.slice/crio-ffb5fa2f4afc52e2a6c30ffafcdb7744d6ea3f1088a3354c2e59dc10ca586b07 WatchSource:0}: Error finding container ffb5fa2f4afc52e2a6c30ffafcdb7744d6ea3f1088a3354c2e59dc10ca586b07: Status 404 returned error can't find the container with id ffb5fa2f4afc52e2a6c30ffafcdb7744d6ea3f1088a3354c2e59dc10ca586b07
Jul 27 17:28:53.548608 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.548588    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55803ecb689b631e773ecebce7207b60.slice/crio-92b68a924da0cd56a7246c0f0cd716f2ea062b0b764f27635dd843bfbcb997da WatchSource:0}: Error finding container 92b68a924da0cd56a7246c0f0cd716f2ea062b0b764f27635dd843bfbcb997da: Status 404 returned error can't find the container with id 92b68a924da0cd56a7246c0f0cd716f2ea062b0b764f27635dd843bfbcb997da
Jul 27 17:28:53.552511 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.552491    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:28:53.614559 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.614479    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:28:53.623247 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.623219    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc14a53_bf9c_4c74_8b11_2953f0979658.slice/crio-f8a7e475a043e19fc5e413b7909c2f711734cb8071d86737fd664d0ac8b63b9a WatchSource:0}: Error finding container f8a7e475a043e19fc5e413b7909c2f711734cb8071d86737fd664d0ac8b63b9a: Status 404 returned error can't find the container with id f8a7e475a043e19fc5e413b7909c2f711734cb8071d86737fd664d0ac8b63b9a
Jul 27 17:28:53.637066 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.637041    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn"
Jul 27 17:28:53.645356 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.645322    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48eca8f_20da_4ade_9817_c88f1a443e89.slice/crio-641588c0571525ef3da0e0dcbfc51bef85afcc9880ad42630f701b50a54b2379 WatchSource:0}: Error finding container 641588c0571525ef3da0e0dcbfc51bef85afcc9880ad42630f701b50a54b2379: Status 404 returned error can't find the container with id 641588c0571525ef3da0e0dcbfc51bef85afcc9880ad42630f701b50a54b2379
Jul 27 17:28:53.653640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.653623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7hkzw"
Jul 27 17:28:53.658814 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.658794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-72mk7"
Jul 27 17:28:53.663966 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.663946    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e52ffd_2d0c_4934_9492_07647e3e1a50.slice/crio-9f879564ac9de18dde1918062da1915dac07626f963bd05559e0807903d0da24 WatchSource:0}: Error finding container 9f879564ac9de18dde1918062da1915dac07626f963bd05559e0807903d0da24: Status 404 returned error can't find the container with id 9f879564ac9de18dde1918062da1915dac07626f963bd05559e0807903d0da24
Jul 27 17:28:53.668652 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.668630    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d73dec_5e9b_48a8_882b_4f67668fb911.slice/crio-d6023f20dcacc6e8a541ab392e1e9c71208ab002268580fea4a1ded48979bd43 WatchSource:0}: Error finding container d6023f20dcacc6e8a541ab392e1e9c71208ab002268580fea4a1ded48979bd43: Status 404 returned error can't find the container with id d6023f20dcacc6e8a541ab392e1e9c71208ab002268580fea4a1ded48979bd43
Jul 27 17:28:53.683376 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.683358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bpmlj"
Jul 27 17:28:53.691204 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.691174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0a41fe_56a4_4dd0_90e0_5ba8ab992da8.slice/crio-c0407f81e206ccb3b963402de50fb54a315801afe9c3eb29a9394e940aa78fad WatchSource:0}: Error finding container c0407f81e206ccb3b963402de50fb54a315801afe9c3eb29a9394e940aa78fad: Status 404 returned error can't find the container with id c0407f81e206ccb3b963402de50fb54a315801afe9c3eb29a9394e940aa78fad
Jul 27 17:28:53.697284 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.697266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gdzng"
Jul 27 17:28:53.701477 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.701349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:28:53.706717 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.706695    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c0dcb3_16e3_46a1_ba47_a91186ce974b.slice/crio-d9482754d18493a8593b7e3422d382a0dc41ff1f64624f1c91be35a240874778 WatchSource:0}: Error finding container d9482754d18493a8593b7e3422d382a0dc41ff1f64624f1c91be35a240874778: Status 404 returned error can't find the container with id d9482754d18493a8593b7e3422d382a0dc41ff1f64624f1c91be35a240874778
Jul 27 17:28:53.709729 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.709704    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5360283_57fc_49d4_87bf_9322bf5e5665.slice/crio-52e01a7a0f43e6320613ac1d7633947a1e0a6368855f4b3d9250525c420fbd6f WatchSource:0}: Error finding container 52e01a7a0f43e6320613ac1d7633947a1e0a6368855f4b3d9250525c420fbd6f: Status 404 returned error can't find the container with id 52e01a7a0f43e6320613ac1d7633947a1e0a6368855f4b3d9250525c420fbd6f
Jul 27 17:28:53.736123 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.736098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w84kh"
Jul 27 17:28:53.741202 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.741182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lw92s"
Jul 27 17:28:53.744371 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.744347    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfaef06_f66e_4e0d_9cae_01a4f177dd39.slice/crio-0126107c679ae2bc179d21192e2acb3c91c31c0152f0487d87836e6a726536bd WatchSource:0}: Error finding container 0126107c679ae2bc179d21192e2acb3c91c31c0152f0487d87836e6a726536bd: Status 404 returned error can't find the container with id 0126107c679ae2bc179d21192e2acb3c91c31c0152f0487d87836e6a726536bd
Jul 27 17:28:53.748777 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:28:53.748758    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c414a3_2b52_4f98_965c_dad0bc417b76.slice/crio-9db6748e1fa0aa1703b2d019cbe9180d721be854e09cb521edcbde5eeaa246f0 WatchSource:0}: Error finding container 9db6748e1fa0aa1703b2d019cbe9180d721be854e09cb521edcbde5eeaa246f0: Status 404 returned error can't find the container with id 9db6748e1fa0aa1703b2d019cbe9180d721be854e09cb521edcbde5eeaa246f0
Jul 27 17:28:53.750295 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.750280    2577 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-lw92s_openshift-network-operator(72c414a3-2b52-4f98-965c-dad0bc417b76): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 27 17:28:53.750339 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.750307    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-lw92s" podUID="72c414a3-2b52-4f98-965c-dad0bc417b76"
Jul 27 17:28:53.991143 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.991096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:53.991309 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:53.991232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:53.991309 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.991266    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:53.991437 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.991351    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:54.991327682 +0000 UTC m=+3.069760089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:53.991437 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.991383    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:53.991555 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:53.991465    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:54.991431993 +0000 UTC m=+3.069864408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:54.044307 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.044271    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:54.092451 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.092128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:54.092451 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:54.092352    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:28:54.092451 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:54.092375    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:28:54.092451 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:54.092389    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:54.092451 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:54.092448    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:55.092428277 +0000 UTC m=+3.170860707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:54.325734 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.325616    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:23:53 +0000 UTC" deadline="2028-03-28 20:18:16.234969171 +0000 UTC"
Jul 27 17:28:54.325734 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.325679    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14642h49m21.90929395s"
Jul 27 17:28:54.490991 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.490929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" event={"ID":"1f24c4d72d8c15a7ea1fde556db108ca","Type":"ContainerStarted","Data":"ffb5fa2f4afc52e2a6c30ffafcdb7744d6ea3f1088a3354c2e59dc10ca586b07"}
Jul 27 17:28:54.497066 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.497032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" event={"ID":"55803ecb689b631e773ecebce7207b60","Type":"ContainerStarted","Data":"92b68a924da0cd56a7246c0f0cd716f2ea062b0b764f27635dd843bfbcb997da"}
Jul 27 17:28:54.499773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.499745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gdzng" event={"ID":"48c0dcb3-16e3-46a1-ba47-a91186ce974b","Type":"ContainerStarted","Data":"d9482754d18493a8593b7e3422d382a0dc41ff1f64624f1c91be35a240874778"}
Jul 27 17:28:54.503651 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.503575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerStarted","Data":"c0407f81e206ccb3b963402de50fb54a315801afe9c3eb29a9394e940aa78fad"}
Jul 27 17:28:54.508017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.507988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6lb2p" event={"ID":"8bc14a53-bf9c-4c74-8b11-2953f0979658","Type":"ContainerStarted","Data":"f8a7e475a043e19fc5e413b7909c2f711734cb8071d86737fd664d0ac8b63b9a"}
Jul 27 17:28:54.510704 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.509935    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lw92s" event={"ID":"72c414a3-2b52-4f98-965c-dad0bc417b76","Type":"ContainerStarted","Data":"9db6748e1fa0aa1703b2d019cbe9180d721be854e09cb521edcbde5eeaa246f0"}
Jul 27 17:28:54.513472 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:54.513053    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-lw92s" podUID="72c414a3-2b52-4f98-965c-dad0bc417b76"
Jul 27 17:28:54.513884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.513827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w84kh" event={"ID":"1cfaef06-f66e-4e0d-9cae-01a4f177dd39","Type":"ContainerStarted","Data":"0126107c679ae2bc179d21192e2acb3c91c31c0152f0487d87836e6a726536bd"}
Jul 27 17:28:54.517769 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.517732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"52e01a7a0f43e6320613ac1d7633947a1e0a6368855f4b3d9250525c420fbd6f"}
Jul 27 17:28:54.520163 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.520094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-72mk7" event={"ID":"75d73dec-5e9b-48a8-882b-4f67668fb911","Type":"ContainerStarted","Data":"d6023f20dcacc6e8a541ab392e1e9c71208ab002268580fea4a1ded48979bd43"}
Jul 27 17:28:54.523198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.523125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7hkzw" event={"ID":"d2e52ffd-2d0c-4934-9492-07647e3e1a50","Type":"ContainerStarted","Data":"9f879564ac9de18dde1918062da1915dac07626f963bd05559e0807903d0da24"}
Jul 27 17:28:54.526726 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:54.526699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn" event={"ID":"e48eca8f-20da-4ade-9817-c88f1a443e89","Type":"ContainerStarted","Data":"641588c0571525ef3da0e0dcbfc51bef85afcc9880ad42630f701b50a54b2379"}
Jul 27 17:28:55.001324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.001285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:55.001519 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.001368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:55.001605 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.001559    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:55.001677 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.001634    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:57.001608237 +0000 UTC m=+5.080040639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:55.001751 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.001713    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:55.001809 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.001752    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:28:57.001740443 +0000 UTC m=+5.080172848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:55.099180 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.099145    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:55.102543 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.102493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:55.102754 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.102711    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:28:55.102754 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.102735    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:28:55.102754 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.102749    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:55.102924 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.102807    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:28:57.102788204 +0000 UTC m=+5.181220612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:55.256833 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.256754    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 17:28:55.326863 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.326819    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 17:23:53 +0000 UTC" deadline="2027-12-29 02:41:25.870571228 +0000 UTC"
Jul 27 17:28:55.326863 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.326861    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12465h12m30.543714226s"
Jul 27 17:28:55.477589 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.477485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:55.477867 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.477616    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:28:55.478059 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.478027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:55.478182 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.478142    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:28:55.478250 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:55.478209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:55.478349 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:55.478300    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:28:57.020444 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:57.020380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:57.020951 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:57.020453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:57.020951 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.020590    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:57.020951 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.020657    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:01.020635365 +0000 UTC m=+9.099067789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:28:57.021146 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.021020    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:57.021146 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.021089    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:01.0210698 +0000 UTC m=+9.099502209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:28:57.121185 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:57.121144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:57.121356 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.121340    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:28:57.121401 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.121360    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:28:57.121401 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.121370    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:57.121502 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.121416    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:29:01.121403075 +0000 UTC m=+9.199835476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:28:57.478001 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:57.477965    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:57.478236 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.478203    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:28:57.478587 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:57.478565    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:57.478698 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:57.478605    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:57.478774 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.478739    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:28:57.478851 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:57.478819    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:28:59.477348 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:59.477300    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:28:59.477809 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:59.477448    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:28:59.477892 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:59.477820    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:28:59.477953 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:59.477911    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:28:59.478005 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:28:59.477959    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:28:59.478052 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:28:59.478036    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:01.054634 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:01.054367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:01.054634 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.054633    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:29:01.055147 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:01.054677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:01.055147 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.054698    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:09.054680328 +0000 UTC m=+17.133112747 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:29:01.055147 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.054746    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:29:01.055147 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.054779    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:09.054769122 +0000 UTC m=+17.133201523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:29:01.155969 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:01.155887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:01.156149 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.156108    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:29:01.156149 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.156135    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:29:01.156149 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.156151    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:29:01.156317 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.156224    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:29:09.156205801 +0000 UTC m=+17.234638217 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:29:01.478455 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:01.477827    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:01.478455 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.477950    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:01.478455 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:01.478255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:01.478455 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.478335    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:01.478455 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:01.478370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:01.478455 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:01.478410    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:03.477729 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:03.477686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:03.478158 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:03.477730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:03.478158 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:03.477734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:03.478158 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:03.477814    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:03.478158 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:03.477908    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:03.478158 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:03.477987    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:05.477482 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:05.477450    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:05.477482 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:05.477482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:05.477944 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:05.477492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:05.477944 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:05.477573    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:05.477944 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:05.477668    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:05.477944 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:05.477731    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:07.477012 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:07.476973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:07.477012 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:07.477008    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:07.477494 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:07.477093    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:07.477494 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:07.477142    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:07.477494 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:07.477222    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:07.477494 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:07.477338    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:09.116537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:09.116458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:09.116537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:09.116515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:09.116990 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.116625    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:29:09.116990 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.116677    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:25.116663224 +0000 UTC m=+33.195095625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:29:09.116990 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.116626    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:29:09.116990 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.116776    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:25.116756778 +0000 UTC m=+33.195189181 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:29:09.217738 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:09.217707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:09.217913 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.217867    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:29:09.217913 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.217886    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:29:09.217913 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.217896    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:29:09.218040 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.217948    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:29:25.217933448 +0000 UTC m=+33.296365849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:29:09.477633 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:09.477603    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:09.477633 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:09.477634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:09.477947 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:09.477604    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:09.477947 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.477743    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:09.477947 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.477888    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:09.478073 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:09.477971    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:11.477301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:11.477258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:11.477301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:11.477276    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:11.477854 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:11.477358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:11.477854 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:11.477367    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:11.477854 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:11.477441    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:11.477854 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:11.477505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:13.478014 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.477784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:13.478464 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.477819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:13.478464 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.477822    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:13.478464 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:13.478130    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:13.478464 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:13.478240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:13.478464 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:13.478334    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:13.577841 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.577797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7hkzw" event={"ID":"d2e52ffd-2d0c-4934-9492-07647e3e1a50","Type":"ContainerStarted","Data":"d2f99c68d4e002931000d2b85bb7af8226f77b5b93b3998d008ed9fcc40a8806"}
Jul 27 17:29:13.579562 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.579537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" event={"ID":"55803ecb689b631e773ecebce7207b60","Type":"ContainerStarted","Data":"aba5b89884fdf288cc2f33faef7d8abf1465fc278552ec86f4028e4ff78c7045"}
Jul 27 17:29:13.581051 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.581026    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gdzng" event={"ID":"48c0dcb3-16e3-46a1-ba47-a91186ce974b","Type":"ContainerStarted","Data":"de32667cc5b96a2b358e6e8559f2412a657e5403c3bab6a71204f46cf5e430e4"}
Jul 27 17:29:13.597157 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.597100    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7hkzw" podStartSLOduration=2.406555425 podStartE2EDuration="21.59708502s" totalImagesPullingTime="19.190529595s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:13.59653618 +0000 UTC m=+21.674968604" watchObservedRunningTime="2026-07-27 17:29:13.59708502 +0000 UTC m=+21.675517442"
Jul 27 17:29:13.613590 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.613552    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gdzng" podStartSLOduration=2.427597813 podStartE2EDuration="21.613526726s" totalImagesPullingTime="19.185928913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:13.613064112 +0000 UTC m=+21.691496534" watchObservedRunningTime="2026-07-27 17:29:13.613526726 +0000 UTC m=+21.691959149"
Jul 27 17:29:13.630928 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:13.630883    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" podStartSLOduration=20.63086612 podStartE2EDuration="20.63086612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:13.630182012 +0000 UTC m=+21.708614434" watchObservedRunningTime="2026-07-27 17:29:13.63086612 +0000 UTC m=+21.709298544"
Jul 27 17:29:14.584109 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.583875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn" event={"ID":"e48eca8f-20da-4ade-9817-c88f1a443e89","Type":"ContainerStarted","Data":"f8ed994ec32b2607adb15fabe33bc66c83fc60d760c1b0a8596f5d9e1ef0c25b"}
Jul 27 17:29:14.585282 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.585260    2577 generic.go:356] "Generic (PLEG): container finished" podID="1f24c4d72d8c15a7ea1fde556db108ca" containerID="bc3c93123065832d5b02a66724e16b61e097b90cf70a0c1c3e1a43d1958afbd9" exitCode=0
Jul 27 17:29:14.585386 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.585325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" event={"ID":"1f24c4d72d8c15a7ea1fde556db108ca","Type":"ContainerDied","Data":"bc3c93123065832d5b02a66724e16b61e097b90cf70a0c1c3e1a43d1958afbd9"}
Jul 27 17:29:14.586902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.586880    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8" containerID="f83a26f08cff392b4767f1796e130d0715b1cfe463f69d7e056ce3da4bb726bd" exitCode=0
Jul 27 17:29:14.586993 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.586946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerDied","Data":"f83a26f08cff392b4767f1796e130d0715b1cfe463f69d7e056ce3da4bb726bd"}
Jul 27 17:29:14.591873 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.591845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6lb2p" event={"ID":"8bc14a53-bf9c-4c74-8b11-2953f0979658","Type":"ContainerStarted","Data":"d0a5cc80242e4bf73f59467a0f87a85e317a98d2111ff802284c3e6f9dc46810"}
Jul 27 17:29:14.593420 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.593393    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lw92s" event={"ID":"72c414a3-2b52-4f98-965c-dad0bc417b76","Type":"ContainerStarted","Data":"fcb18f056516cbb40346a4dd11f50dcdaf62530f547845decbb11e92144e5213"}
Jul 27 17:29:14.594739 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.594719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w84kh" event={"ID":"1cfaef06-f66e-4e0d-9cae-01a4f177dd39","Type":"ContainerStarted","Data":"fcf2968872b35d75db9da032656654271499961e1d9b6c92bbcd15ee11f9b8d3"}
Jul 27 17:29:14.597406 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597388    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:29:14.597731 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597710    2577 generic.go:356] "Generic (PLEG): container finished" podID="c5360283-57fc-49d4-87bf-9322bf5e5665" containerID="d65e5143d331af200d5d996d48284c548eafccef16b56a64c4a06c47e6820b13" exitCode=1
Jul 27 17:29:14.597794 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"a5e579a3204563e2a1acbec722497e115f7352ca67bf2b8d1e135b9df463bb37"}
Jul 27 17:29:14.597847 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597835    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"8f13c25cbc9d3b624edbb33ced491d466cf66ace4bde3424d62d41a20a460d61"}
Jul 27 17:29:14.597887 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"24a05cf05d53a680dbba25e4e8046fd888bf904b6607c4aec7b9b24ed7b0534b"}
Jul 27 17:29:14.597936 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"784e4c0819ac0cda86e7f23a82539a551424da5477d12c3b756cfb6df6b9af8d"}
Jul 27 17:29:14.597973 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"0ec888d88128f9ca95147fb1f7c0991fcde59a184a12abf2f57671cb36eb7b0d"}
Jul 27 17:29:14.597973 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.597966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerDied","Data":"d65e5143d331af200d5d996d48284c548eafccef16b56a64c4a06c47e6820b13"}
Jul 27 17:29:14.598991 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.598974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-72mk7" event={"ID":"75d73dec-5e9b-48a8-882b-4f67668fb911","Type":"ContainerStarted","Data":"ac1200a7d95c2ac61950fc6c38e6a68c4088a050fd08f9eaccb48657a19bbdd5"}
Jul 27 17:29:14.600153 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.600138    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:29:14.600642 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.600626    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:29:14.638261 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.638211    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lw92s" podStartSLOduration=22.638191335 podStartE2EDuration="22.638191335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-27 17:29:14.637669248 +0000 UTC m=+22.716101672" watchObservedRunningTime="2026-07-27 17:29:14.638191335 +0000 UTC m=+22.716623759"
Jul 27 17:29:14.654707 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.654661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6lb2p" podStartSLOduration=14.558592296 podStartE2EDuration="22.654641031s" totalImagesPullingTime="8.096048735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:14.654021905 +0000 UTC m=+22.732454329" watchObservedRunningTime="2026-07-27 17:29:14.654641031 +0000 UTC m=+22.733073455"
Jul 27 17:29:14.682074 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.682021    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w84kh" podStartSLOduration=7.796453551 podStartE2EDuration="22.682002853s" totalImagesPullingTime="14.885549302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:14.668226706 +0000 UTC m=+22.746659130" watchObservedRunningTime="2026-07-27 17:29:14.682002853 +0000 UTC m=+22.760435293"
Jul 27 17:29:14.682577 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.682540    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-72mk7" podStartSLOduration=11.400427468 podStartE2EDuration="22.682527771s" totalImagesPullingTime="11.282100303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:14.681556544 +0000 UTC m=+22.759988966" watchObservedRunningTime="2026-07-27 17:29:14.682527771 +0000 UTC m=+22.760960195"
Jul 27 17:29:14.775483 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.775450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:14.957853 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:14.957827    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 17:29:15.324526 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.324491    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:15.349865 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.349739    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T17:29:14.95784546Z","UUID":"f2094477-a608-42a5-b546-78ae8bb10125","Handler":null,"Name":"","Endpoint":""}
Jul 27 17:29:15.351665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.351636    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 17:29:15.351807 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.351684    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 17:29:15.477809 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.477778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:15.478007 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.477823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:15.478007 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:15.477946    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:15.478184 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.478168    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:15.478317 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:15.478267    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:15.478455 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:15.478341    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:15.604461 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.604365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" event={"ID":"1f24c4d72d8c15a7ea1fde556db108ca","Type":"ContainerStarted","Data":"e1b057f433d81a7d46c0190166e9a1b2a9373080e604363db73b53aa23b60c20"}
Jul 27 17:29:15.607363 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.607337    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn" event={"ID":"e48eca8f-20da-4ade-9817-c88f1a443e89","Type":"ContainerStarted","Data":"6f8d42811dbdebc0bc0208e45a7a5cf5ac1b77465097400b2c4fa87699551d42"}
Jul 27 17:29:15.619743 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:15.619682    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" podStartSLOduration=22.619656967 podStartE2EDuration="22.619656967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:15.619300107 +0000 UTC m=+23.697732540" watchObservedRunningTime="2026-07-27 17:29:15.619656967 +0000 UTC m=+23.698089392"
Jul 27 17:29:16.613946 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:16.613715    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:29:16.614496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:16.614285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"fcab23812ae69c98ed8608a0bbec8bff88c7a121e0081075cf19e459600a1f63"}
Jul 27 17:29:16.616713 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:16.616638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn" event={"ID":"e48eca8f-20da-4ade-9817-c88f1a443e89","Type":"ContainerStarted","Data":"bd2274b6cb89214f9cac558c6ad3658b279fc620b739b561551342e7295559c4"}
Jul 27 17:29:16.616713 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:16.616707    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 27 17:29:16.633207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:16.633154    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m82gn" podStartSLOduration=3.203527281 podStartE2EDuration="24.633133751s" totalImagesPullingTime="21.42960647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:16.633047364 +0000 UTC m=+24.711479788" watchObservedRunningTime="2026-07-27 17:29:16.633133751 +0000 UTC m=+24.711566176"
Jul 27 17:29:16.754836 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:16.754801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:17.477242 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:17.477204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:17.477242 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:17.477242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:17.477540 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:17.477326    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:17.477540 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:17.477339    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:17.477540 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:17.477406    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:17.477540 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:17.477509    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:18.510228 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:18.510186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:18.567856 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:18.567828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:19.477814 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.477639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:19.478011 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.477639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:19.478011 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:19.477883    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:19.478011 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:19.477976    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:19.478011 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.477639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:19.478179 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:19.478075    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:19.625921 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.625898    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:29:19.626597 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.626251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"a3a7d53c542d743f768e2b7942a7816302658ec499fc9f736556d66fad7ac3c3"}
Jul 27 17:29:19.626597 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.626531    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:29:19.626691 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.626643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:29:19.626691 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.626679    2577 scope.go:117] "RemoveContainer" containerID="d65e5143d331af200d5d996d48284c548eafccef16b56a64c4a06c47e6820b13"
Jul 27 17:29:19.628262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.628241    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8" containerID="6e220e51d2fe2160f5de149398e250eb3a77ddddd54f793e3159c73d879e9e11" exitCode=0
Jul 27 17:29:19.628262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.628269    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerDied","Data":"6e220e51d2fe2160f5de149398e250eb3a77ddddd54f793e3159c73d879e9e11"}
Jul 27 17:29:19.643052 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:19.643031    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:29:20.632980 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.632933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerStarted","Data":"2b01a9b9abb74c5d2c3cf2886792456822536c499a5208b997f4873e29a5e8da"}
Jul 27 17:29:20.637172 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.637152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:29:20.637611 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.637584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" event={"ID":"c5360283-57fc-49d4-87bf-9322bf5e5665","Type":"ContainerStarted","Data":"4bbd96f3fd3de037675992f2a82d6e902808427c80872dc1f19f3384603d84ae"}
Jul 27 17:29:20.637870 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.637847    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:29:20.653342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.653320    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:29:20.668860 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.668674    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j" podStartSLOduration=8.930102699 podStartE2EDuration="28.668659137s" totalImagesPullingTime="19.738556438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:20.667190431 +0000 UTC m=+28.745622853" watchObservedRunningTime="2026-07-27 17:29:20.668659137 +0000 UTC m=+28.747091560"
Jul 27 17:29:20.850654 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.850583    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qxkrc"]
Jul 27 17:29:20.850801 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.850696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:20.850801 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:20.850782    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:20.853200 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.853177    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29lnk"]
Jul 27 17:29:20.853314 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.853288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:20.853404 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:20.853382    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:20.867031 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.867007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fncdg"]
Jul 27 17:29:20.867125 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:20.867090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:20.867197 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:20.867163    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:21.642186 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:21.642154    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8" containerID="2b01a9b9abb74c5d2c3cf2886792456822536c499a5208b997f4873e29a5e8da" exitCode=0
Jul 27 17:29:21.642592 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:21.642236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerDied","Data":"2b01a9b9abb74c5d2c3cf2886792456822536c499a5208b997f4873e29a5e8da"}
Jul 27 17:29:22.478307 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:22.478279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:22.478441 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:22.478368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:22.478441 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:22.478382    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:22.478531 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:22.478505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:22.478574 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:22.478543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:22.478634 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:22.478617    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:22.646748 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:22.646715    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8" containerID="c33b6099d3224b8026ee0bbe78b22f8a7617d0998f0d403120692095e0ab8ba8" exitCode=0
Jul 27 17:29:22.647302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:22.646800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerDied","Data":"c33b6099d3224b8026ee0bbe78b22f8a7617d0998f0d403120692095e0ab8ba8"}
Jul 27 17:29:23.430215 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:23.430176    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:29:23.430415 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:23.430294    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 27 17:29:23.430814 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:23.430796    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6lb2p"
Jul 27 17:29:23.492160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:23.492128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:24.476928 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:24.476899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:24.477642 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:24.476899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:24.477642 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:24.477015    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:24.477642 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:24.477102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:24.477642 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:24.476898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:24.477642 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:24.477165    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:25.135557 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:25.135516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:25.135742 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:25.135597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:25.135742 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.135637    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:29:25.135742 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.135690    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 17:29:25.135742 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.135709    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:57.135690047 +0000 UTC m=+65.214122462 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 17:29:25.135742 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.135735    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret podName:c9cb2437-7ae9-44d4-8f5f-927e38a41769 nodeName:}" failed. No retries permitted until 2026-07-27 17:29:57.135723491 +0000 UTC m=+65.214155892 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret") pod "global-pull-secret-syncer-qxkrc" (UID: "c9cb2437-7ae9-44d4-8f5f-927e38a41769") : object "kube-system"/"original-pull-secret" not registered
Jul 27 17:29:25.235951 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:25.235914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:25.236133 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.236066    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 17:29:25.236133 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.236081    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 17:29:25.236133 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.236091    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gsfwj for pod openshift-network-diagnostics/network-check-target-fncdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:29:25.236296 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:25.236150    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj podName:c1e7b60f-8feb-4bb6-8446-ca527cd7475a nodeName:}" failed. No retries permitted until 2026-07-27 17:29:57.236132742 +0000 UTC m=+65.314565147 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gsfwj" (UniqueName: "kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj") pod "network-check-target-fncdg" (UID: "c1e7b60f-8feb-4bb6-8446-ca527cd7475a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 17:29:26.477527 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:26.477492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:26.478023 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:26.477496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:26.478023 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:26.477627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:26.478023 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:26.477620    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:29:26.478023 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:26.477748    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qxkrc" podUID="c9cb2437-7ae9-44d4-8f5f-927e38a41769"
Jul 27 17:29:26.478023 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:26.477831    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fncdg" podUID="c1e7b60f-8feb-4bb6-8446-ca527cd7475a"
Jul 27 17:29:27.297360 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.297329    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeReady"
Jul 27 17:29:27.297549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.297540    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 17:29:27.340902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.340867    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t4lwh"]
Jul 27 17:29:27.349362 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.349338    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dmr84"]
Jul 27 17:29:27.349517 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.349469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.352260 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.352229    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 17:29:27.353173 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.352643    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 17:29:27.353173 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.352887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x7wp7\""
Jul 27 17:29:27.353585 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.353531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t4lwh"]
Jul 27 17:29:27.353585 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.353627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.356106 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.356080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dmr84"]
Jul 27 17:29:27.356487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.356468    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 17:29:27.356948 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.356927    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 17:29:27.357890 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.357717    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 17:29:27.357999 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.357930    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6fr89\""
Jul 27 17:29:27.453784 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.453744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-tmp-dir\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.453958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.453792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32b34b39-946f-4c7b-855b-7905b1b84903-cert\") pod \"ingress-canary-dmr84\" (UID: \"32b34b39-946f-4c7b-855b-7905b1b84903\") " pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.453958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.453833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.453958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.453852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbhtk\" (UniqueName: \"kubernetes.io/projected/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-kube-api-access-dbhtk\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.453958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.453876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-config-volume\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.453958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.453922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvr4\" (UniqueName: \"kubernetes.io/projected/32b34b39-946f-4c7b-855b-7905b1b84903-kube-api-access-nxvr4\") pod \"ingress-canary-dmr84\" (UID: \"32b34b39-946f-4c7b-855b-7905b1b84903\") " pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.456889 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.456856    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:27.511514 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.511456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:27.554996 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.554934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-tmp-dir\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.554996 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.554964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32b34b39-946f-4c7b-855b-7905b1b84903-cert\") pod \"ingress-canary-dmr84\" (UID: \"32b34b39-946f-4c7b-855b-7905b1b84903\") " pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.555193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.554998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.555193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.555025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbhtk\" (UniqueName: \"kubernetes.io/projected/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-kube-api-access-dbhtk\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.555193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.555051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-config-volume\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.555193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.555085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvr4\" (UniqueName: \"kubernetes.io/projected/32b34b39-946f-4c7b-855b-7905b1b84903-kube-api-access-nxvr4\") pod \"ingress-canary-dmr84\" (UID: \"32b34b39-946f-4c7b-855b-7905b1b84903\") " pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.555193 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:27.555113    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:27.555193 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:27.555180    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:29:28.055158569 +0000 UTC m=+36.133591002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:29:27.555623 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.555596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-tmp-dir\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.555854 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.555834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-config-volume\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.559904 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.559749    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32b34b39-946f-4c7b-855b-7905b1b84903-cert\") pod \"ingress-canary-dmr84\" (UID: \"32b34b39-946f-4c7b-855b-7905b1b84903\") " pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.564193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.564159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbhtk\" (UniqueName: \"kubernetes.io/projected/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-kube-api-access-dbhtk\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:27.564323 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.564298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvr4\" (UniqueName: \"kubernetes.io/projected/32b34b39-946f-4c7b-855b-7905b1b84903-kube-api-access-nxvr4\") pod \"ingress-canary-dmr84\" (UID: \"32b34b39-946f-4c7b-855b-7905b1b84903\") " pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.668750 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.668723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dmr84"
Jul 27 17:29:27.890372 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:27.890343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dmr84"]
Jul 27 17:29:27.919729 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:29:27.919705    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b34b39_946f_4c7b_855b_7905b1b84903.slice/crio-45f425e0111d4fffe62bca5cbedbcba30b287641420b560dd1d694e7e93abcb9 WatchSource:0}: Error finding container 45f425e0111d4fffe62bca5cbedbcba30b287641420b560dd1d694e7e93abcb9: Status 404 returned error can't find the container with id 45f425e0111d4fffe62bca5cbedbcba30b287641420b560dd1d694e7e93abcb9
Jul 27 17:29:28.059524 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.059502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:28.059631 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:28.059619    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:28.059674 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:28.059664    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:29:29.059651451 +0000 UTC m=+37.138083852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:29:28.424736 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.424649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:28.477480 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.477447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:28.477608 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.477447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:28.477695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.477462    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:28.478550 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.478529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:28.482085 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.482059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qwvcb\""
Jul 27 17:29:28.482085 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.482073    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:29:28.482243 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.482076    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:29:28.482243 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.482097    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:29:28.482243 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.482061    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:29:28.482243 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.482076    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zkh28\""
Jul 27 17:29:28.663592 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.663555    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8" containerID="d5f367ff3630820d8f6624b08b143d62c3c90c1ed9a218c1b7958d00d600adc3" exitCode=0
Jul 27 17:29:28.664053 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.663640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerDied","Data":"d5f367ff3630820d8f6624b08b143d62c3c90c1ed9a218c1b7958d00d600adc3"}
Jul 27 17:29:28.664719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:28.664697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dmr84" event={"ID":"32b34b39-946f-4c7b-855b-7905b1b84903","Type":"ContainerStarted","Data":"45f425e0111d4fffe62bca5cbedbcba30b287641420b560dd1d694e7e93abcb9"}
Jul 27 17:29:29.066659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:29.066619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:29.066825 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:29.066780    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:29.066886 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:29.066850    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:29:31.066830623 +0000 UTC m=+39.145263032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:29:29.670684 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:29.670647    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8" containerID="915250c58b6199babc4c1e1d15c4dd9ddb9a3d7e7dffa52d76ad7279cf3d8c12" exitCode=0
Jul 27 17:29:29.670684 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:29.670673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerDied","Data":"915250c58b6199babc4c1e1d15c4dd9ddb9a3d7e7dffa52d76ad7279cf3d8c12"}
Jul 27 17:29:30.678472 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:30.678220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" event={"ID":"9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8","Type":"ContainerStarted","Data":"9becfb08a23d578f573fc3ebf348fe57c756185711513cb7637934d54f452ad1"}
Jul 27 17:29:30.701081 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:30.701033    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bpmlj" podStartSLOduration=8.570649421 podStartE2EDuration="38.701002207s" totalImagesPullingTime="30.130352786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:30.699961665 +0000 UTC m=+38.778394088" watchObservedRunningTime="2026-07-27 17:29:30.701002207 +0000 UTC m=+38.779434629"
Jul 27 17:29:30.821140 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:30.821108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:30.880902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:30.880870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:31.084395 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:31.084365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:31.084540 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:31.084523    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:31.084600 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:31.084590    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:29:35.084571701 +0000 UTC m=+43.163004102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:29:31.339093 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:31.339008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:31.388126 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:31.388095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:31.681680 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:31.681599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dmr84" event={"ID":"32b34b39-946f-4c7b-855b-7905b1b84903","Type":"ContainerStarted","Data":"a45389c9faf58c3f46fb57a38f837b16b05ecd7501e13bde225215cedcbd473f"}
Jul 27 17:29:31.696926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:31.696880    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dmr84" podStartSLOduration=1.514357738 podStartE2EDuration="4.696865366s" totalImagesPullingTime="3.182507628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:29:31.695924179 +0000 UTC m=+39.774356614" watchObservedRunningTime="2026-07-27 17:29:31.696865366 +0000 UTC m=+39.775297789"
Jul 27 17:29:35.111175 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:35.111141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:35.111557 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:35.111248    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:35.111557 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:35.111303    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:29:43.111286315 +0000 UTC m=+51.189718716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:29:39.752804 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:39.752774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:39.801565 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:39.801532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:43.165614 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:43.165580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:43.165996 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:43.165741    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:43.165996 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:43.165813    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:29:59.165796663 +0000 UTC m=+67.244229063 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:29:52.657440 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:52.657395    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nld5j"
Jul 27 17:29:52.779739 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:52.779706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:52.835236 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:52.835205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:29:57.163713 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.163671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:29:57.164096 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.163737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:57.166728 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.166710    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 17:29:57.166775 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.166730    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 17:29:57.173932 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:57.173911    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 17:29:57.173994 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:57.173983    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:31:01.173962596 +0000 UTC m=+129.252395002 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : secret "metrics-daemon-secret" not found
Jul 27 17:29:57.177518 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.177498    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c9cb2437-7ae9-44d4-8f5f-927e38a41769-original-pull-secret\") pod \"global-pull-secret-syncer-qxkrc\" (UID: \"c9cb2437-7ae9-44d4-8f5f-927e38a41769\") " pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:57.264268 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.264229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:57.266981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.266963    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 17:29:57.277124 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.277102    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 17:29:57.287118 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.287097    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qxkrc"
Jul 27 17:29:57.287453 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.287419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsfwj\" (UniqueName: \"kubernetes.io/projected/c1e7b60f-8feb-4bb6-8446-ca527cd7475a-kube-api-access-gsfwj\") pod \"network-check-target-fncdg\" (UID: \"c1e7b60f-8feb-4bb6-8446-ca527cd7475a\") " pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:57.299453 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.299424    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zkh28\""
Jul 27 17:29:57.307643 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.307611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:29:57.414362 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.414293    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qxkrc"]
Jul 27 17:29:57.418221 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:29:57.418192    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9cb2437_7ae9_44d4_8f5f_927e38a41769.slice/crio-78ea25039bf9deaee55b0499675bd653ec582ffa1906b8c73156bebb68900f70 WatchSource:0}: Error finding container 78ea25039bf9deaee55b0499675bd653ec582ffa1906b8c73156bebb68900f70: Status 404 returned error can't find the container with id 78ea25039bf9deaee55b0499675bd653ec582ffa1906b8c73156bebb68900f70
Jul 27 17:29:57.431002 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.430980    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fncdg"]
Jul 27 17:29:57.433834 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:29:57.433809    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e7b60f_8feb_4bb6_8446_ca527cd7475a.slice/crio-dec6c11ee1957b739fcd0e05a98853015454e58537336882633c2d485aa014d7 WatchSource:0}: Error finding container dec6c11ee1957b739fcd0e05a98853015454e58537336882633c2d485aa014d7: Status 404 returned error can't find the container with id dec6c11ee1957b739fcd0e05a98853015454e58537336882633c2d485aa014d7
Jul 27 17:29:57.743953 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.743919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fncdg" event={"ID":"c1e7b60f-8feb-4bb6-8446-ca527cd7475a","Type":"ContainerStarted","Data":"dec6c11ee1957b739fcd0e05a98853015454e58537336882633c2d485aa014d7"}
Jul 27 17:29:57.744992 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:57.744973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qxkrc" event={"ID":"c9cb2437-7ae9-44d4-8f5f-927e38a41769","Type":"ContainerStarted","Data":"78ea25039bf9deaee55b0499675bd653ec582ffa1906b8c73156bebb68900f70"}
Jul 27 17:29:59.175591 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:29:59.175549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:29:59.176034 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:59.175703    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:29:59.176034 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:29:59.175773    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:30:31.175756832 +0000 UTC m=+99.254189236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:30:00.754191 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:00.754154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fncdg" event={"ID":"c1e7b60f-8feb-4bb6-8446-ca527cd7475a","Type":"ContainerStarted","Data":"330f4c5114fe53d16eea1488f8a1b489a0ad6cc2c5a257798037574fc991d9f0"}
Jul 27 17:30:00.754702 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:00.754269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:30:00.769541 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:00.769494    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fncdg" podStartSLOduration=66.114931363 podStartE2EDuration="1m8.769477529s" totalImagesPullingTime="2.654546166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:00.769245116 +0000 UTC m=+68.847677540" watchObservedRunningTime="2026-07-27 17:30:00.769477529 +0000 UTC m=+68.847909953"
Jul 27 17:30:03.762997 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:03.762959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qxkrc" event={"ID":"c9cb2437-7ae9-44d4-8f5f-927e38a41769","Type":"ContainerStarted","Data":"be2c41ed6ff43277d240524327145829a9cc9c4216dd6a40e91f8bde782e677b"}
Jul 27 17:30:03.777298 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:03.777249    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qxkrc" podStartSLOduration=65.989443583 podStartE2EDuration="1m11.777234401s" totalImagesPullingTime="5.787790818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:30:03.776226994 +0000 UTC m=+71.854659416" watchObservedRunningTime="2026-07-27 17:30:03.777234401 +0000 UTC m=+71.855666889"
Jul 27 17:30:31.188128 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:31.188090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:30:31.188560 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:30:31.188213    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:30:31.188560 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:30:31.188276    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:31:35.188262379 +0000 UTC m=+163.266694780 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:30:31.759149 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:31.759121    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fncdg"
Jul 27 17:30:54.642687 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:30:54.642653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:31:01.184401 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:31:01.184351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:31:01.184836 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:31:01.184532    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 17:31:01.184836 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:31:01.184600    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs podName:c7d5444a-98d9-4fe7-8017-a35216c24332 nodeName:}" failed. No retries permitted until 2026-07-27 17:33:03.184583289 +0000 UTC m=+251.263015691 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs") pod "network-metrics-daemon-29lnk" (UID: "c7d5444a-98d9-4fe7-8017-a35216c24332") : secret "metrics-daemon-secret" not found
Jul 27 17:31:30.363115 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:31:30.363074    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t4lwh" podUID="42cb6f4f-5199-41c8-8f79-4c847ede7bcd"
Jul 27 17:31:30.964625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:31:30.964596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:31:31.493078 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:31:31.493044    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-29lnk" podUID="c7d5444a-98d9-4fe7-8017-a35216c24332"
Jul 27 17:31:35.212744 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:31:35.212691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:31:35.213148 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:31:35.212824    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 17:31:35.213148 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:31:35.212878    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls podName:42cb6f4f-5199-41c8-8f79-4c847ede7bcd nodeName:}" failed. No retries permitted until 2026-07-27 17:33:37.212863846 +0000 UTC m=+285.291296247 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls") pod "dns-default-t4lwh" (UID: "42cb6f4f-5199-41c8-8f79-4c847ede7bcd") : secret "dns-default-metrics-tls" not found
Jul 27 17:31:44.477342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:31:44.477247    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:32:38.475982 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.475948    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"]
Jul 27 17:32:38.478042 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.478020    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.481721 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.481696    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.482523 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.482505    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wcwl9\""
Jul 27 17:32:38.485133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.484454    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.485133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.484484    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 17:32:38.485133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.484925    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 17:32:38.485945 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.485862    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jvqln"]
Jul 27 17:32:38.487821 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.487804    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"]
Jul 27 17:32:38.487940 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.487926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.489738 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.489723    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"]
Jul 27 17:32:38.489838 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.489821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"
Jul 27 17:32:38.491567 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.491552    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b46967c9f-ptz2d"]
Jul 27 17:32:38.491690 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.491673    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:38.492403 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.492386    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.493506 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.493489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.493599 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.493584    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.493771 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.493752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 17:32:38.493841 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.493817    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.494671 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.494655    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 17:32:38.495728 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.495711    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.495822 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.495727    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kq7kc\""
Jul 27 17:32:38.495890 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.495825    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qv5t4\""
Jul 27 17:32:38.495890 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.495843    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.496190 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.496174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 17:32:38.496190 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.496187    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.496644 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.496628    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 17:32:38.497719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.497260    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"]
Jul 27 17:32:38.497719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.497362    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 17:32:38.497719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.497614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 17:32:38.497719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.497647    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-x7pv8\""
Jul 27 17:32:38.498025 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.497841    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.498025 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.497965    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.498313 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.498274    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-q5f5c\""
Jul 27 17:32:38.498463 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.498401    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jvqln"]
Jul 27 17:32:38.498708 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.498691    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 17:32:38.501536 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.501516    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 17:32:38.503862 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.503839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"]
Jul 27 17:32:38.510496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.510477    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b46967c9f-ptz2d"]
Jul 27 17:32:38.512347 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.512327    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"]
Jul 27 17:32:38.515998 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.515973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zwmr\" (UniqueName: \"kubernetes.io/projected/3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4-kube-api-access-2zwmr\") pod \"volume-data-source-validator-549c6d54cf-mdrqd\" (UID: \"3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"
Jul 27 17:32:38.516101 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-stats-auth\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.516101 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcmxs\" (UniqueName: \"kubernetes.io/projected/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-kube-api-access-vcmxs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.516101 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6797ac-2bcb-45b5-94a0-6359caeff863-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.516358 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6797ac-2bcb-45b5-94a0-6359caeff863-serving-cert\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.516358 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-default-certificate\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.516358 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516279    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.516358 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/024b9af0-76c5-41b8-87d7-a763b6f1a919-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.516677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:38.516677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cws8x\" (UniqueName: \"kubernetes.io/projected/024b9af0-76c5-41b8-87d7-a763b6f1a919-kube-api-access-cws8x\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.516677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqgxg\" (UniqueName: \"kubernetes.io/projected/ab6797ac-2bcb-45b5-94a0-6359caeff863-kube-api-access-pqgxg\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.516677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6797ac-2bcb-45b5-94a0-6359caeff863-tmp\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.516677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ab6797ac-2bcb-45b5-94a0-6359caeff863-snapshots\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.516677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6797ac-2bcb-45b5-94a0-6359caeff863-service-ca-bundle\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.516945 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.516945 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjtn4\" (UniqueName: \"kubernetes.io/projected/0b94f98e-f593-4af6-8200-98ee01580a48-kube-api-access-jjtn4\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:38.516945 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.516754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.571336 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.571286    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"]
Jul 27 17:32:38.575693 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.575668    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-pwdzm"]
Jul 27 17:32:38.575828 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.575784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.577897 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.577878    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 17:32:38.578120 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.578100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.578210 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.578190    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.578276 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.578237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.578327 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.578191    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 17:32:38.578327 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.578240    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6v78l\""
Jul 27 17:32:38.580111 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.580094    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 17:32:38.580412 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.580368    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 17:32:38.580586 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.580570    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 17:32:38.580651 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.580570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lgtqz\""
Jul 27 17:32:38.580918 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.580905    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 17:32:38.584758 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.584737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"]
Jul 27 17:32:38.585747 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.585732    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 17:32:38.597511 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.597490    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-pwdzm"]
Jul 27 17:32:38.617690 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.617864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcmxs\" (UniqueName: \"kubernetes.io/projected/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-kube-api-access-vcmxs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.617864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-stats-auth\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.617864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfmzk\" (UniqueName: \"kubernetes.io/projected/79d2f48c-0254-434b-a3bf-455dd45f270a-kube-api-access-cfmzk\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.617864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.617864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-serving-cert\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.617901    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.617941    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:39.117924335 +0000 UTC m=+227.196356736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : configmap references non-existent config key: service-ca.crt
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.617957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xk9t\" (UniqueName: \"kubernetes.io/projected/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-kube-api-access-4xk9t\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.617967    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls podName:024b9af0-76c5-41b8-87d7-a763b6f1a919 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:39.117954012 +0000 UTC m=+227.196386413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-4pw8b" (UID: "024b9af0-76c5-41b8-87d7-a763b6f1a919") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/024b9af0-76c5-41b8-87d7-a763b6f1a919-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79d2f48c-0254-434b-a3bf-455dd45f270a-config\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.618115    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cws8x\" (UniqueName: \"kubernetes.io/projected/024b9af0-76c5-41b8-87d7-a763b6f1a919-kube-api-access-cws8x\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.618132 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.618142    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls podName:0b94f98e-f593-4af6-8200-98ee01580a48 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:39.118134192 +0000 UTC m=+227.196566594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls") pod "cluster-samples-operator-6748489656-tr78b" (UID: "0b94f98e-f593-4af6-8200-98ee01580a48") : secret "samples-operator-tls" not found
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-config\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6797ac-2bcb-45b5-94a0-6359caeff863-tmp\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79d2f48c-0254-434b-a3bf-455dd45f270a-trusted-ca\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ab6797ac-2bcb-45b5-94a0-6359caeff863-snapshots\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79d2f48c-0254-434b-a3bf-455dd45f270a-serving-cert\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6797ac-2bcb-45b5-94a0-6359caeff863-service-ca-bundle\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjtn4\" (UniqueName: \"kubernetes.io/projected/0b94f98e-f593-4af6-8200-98ee01580a48-kube-api-access-jjtn4\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zwmr\" (UniqueName: \"kubernetes.io/projected/3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4-kube-api-access-2zwmr\") pod \"volume-data-source-validator-549c6d54cf-mdrqd\" (UID: \"3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqgxg\" (UniqueName: \"kubernetes.io/projected/ab6797ac-2bcb-45b5-94a0-6359caeff863-kube-api-access-pqgxg\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6797ac-2bcb-45b5-94a0-6359caeff863-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6797ac-2bcb-45b5-94a0-6359caeff863-serving-cert\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-default-certificate\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab6797ac-2bcb-45b5-94a0-6359caeff863-tmp\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.618695 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.619342 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.618745    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:32:38.619342 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:38.618787    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:39.118772429 +0000 UTC m=+227.197204840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : secret "router-metrics-certs-default" not found
Jul 27 17:32:38.619342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/024b9af0-76c5-41b8-87d7-a763b6f1a919-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.619342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ab6797ac-2bcb-45b5-94a0-6359caeff863-snapshots\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.619342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.618907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6797ac-2bcb-45b5-94a0-6359caeff863-service-ca-bundle\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.619342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.619303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6797ac-2bcb-45b5-94a0-6359caeff863-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.620709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.620664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-stats-auth\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.621230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.621208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-default-certificate\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.621571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.621551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab6797ac-2bcb-45b5-94a0-6359caeff863-serving-cert\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.628657 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.628622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cws8x\" (UniqueName: \"kubernetes.io/projected/024b9af0-76c5-41b8-87d7-a763b6f1a919-kube-api-access-cws8x\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:38.629494 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.629469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjtn4\" (UniqueName: \"kubernetes.io/projected/0b94f98e-f593-4af6-8200-98ee01580a48-kube-api-access-jjtn4\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:38.629627 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.629590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zwmr\" (UniqueName: \"kubernetes.io/projected/3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4-kube-api-access-2zwmr\") pod \"volume-data-source-validator-549c6d54cf-mdrqd\" (UID: \"3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"
Jul 27 17:32:38.629759 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.629740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqgxg\" (UniqueName: \"kubernetes.io/projected/ab6797ac-2bcb-45b5-94a0-6359caeff863-kube-api-access-pqgxg\") pod \"insights-operator-5885fdb759-jvqln\" (UID: \"ab6797ac-2bcb-45b5-94a0-6359caeff863\") " pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.629895 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.629875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcmxs\" (UniqueName: \"kubernetes.io/projected/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-kube-api-access-vcmxs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:38.691282 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.691251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:38.719342 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-config\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.719416 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79d2f48c-0254-434b-a3bf-455dd45f270a-trusted-ca\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.719416 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79d2f48c-0254-434b-a3bf-455dd45f270a-serving-cert\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.719521 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfmzk\" (UniqueName: \"kubernetes.io/projected/79d2f48c-0254-434b-a3bf-455dd45f270a-kube-api-access-cfmzk\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.719565 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-serving-cert\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.719612 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xk9t\" (UniqueName: \"kubernetes.io/projected/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-kube-api-access-4xk9t\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.719746 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79d2f48c-0254-434b-a3bf-455dd45f270a-config\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.719957 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.719932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-config\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.720817 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.720791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79d2f48c-0254-434b-a3bf-455dd45f270a-config\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.720924 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.720906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79d2f48c-0254-434b-a3bf-455dd45f270a-trusted-ca\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.721747 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.721726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79d2f48c-0254-434b-a3bf-455dd45f270a-serving-cert\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.721825 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.721755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-serving-cert\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.727501 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.727445    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfmzk\" (UniqueName: \"kubernetes.io/projected/79d2f48c-0254-434b-a3bf-455dd45f270a-kube-api-access-cfmzk\") pod \"console-operator-584dcb6b54-pwdzm\" (UID: \"79d2f48c-0254-434b-a3bf-455dd45f270a\") " pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.727715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.727700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xk9t\" (UniqueName: \"kubernetes.io/projected/05de6f60-7c44-4ad5-9fac-8101d7d0c3ac-kube-api-access-4xk9t\") pod \"service-ca-operator-5d6c75f84f-jht7q\" (UID: \"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.779141 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.779099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:38.799788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.799764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-jvqln"
Jul 27 17:32:38.806805 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.806786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"
Jul 27 17:32:38.887041 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.886790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"
Jul 27 17:32:38.892268 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.892210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:38.929977 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.929706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd"]
Jul 27 17:32:38.934440 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:38.934383    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1da1a1_2b28_45bf_9cac_5ed5c01b8cc4.slice/crio-1aab560bc756321e8f9de56f78f93180166b20e4da46fb1868f89c530da81167 WatchSource:0}: Error finding container 1aab560bc756321e8f9de56f78f93180166b20e4da46fb1868f89c530da81167: Status 404 returned error can't find the container with id 1aab560bc756321e8f9de56f78f93180166b20e4da46fb1868f89c530da81167
Jul 27 17:32:38.954442 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:38.954354    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jvqln"]
Jul 27 17:32:38.960936 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:38.960532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6797ac_2bcb_45b5_94a0_6359caeff863.slice/crio-19603db53bdeed29ca4d35939f5271b48b1d5f923bf8635206bed7cb347a29ae WatchSource:0}: Error finding container 19603db53bdeed29ca4d35939f5271b48b1d5f923bf8635206bed7cb347a29ae: Status 404 returned error can't find the container with id 19603db53bdeed29ca4d35939f5271b48b1d5f923bf8635206bed7cb347a29ae
Jul 27 17:32:39.019262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.019232    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q"]
Jul 27 17:32:39.022586 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:39.022551    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05de6f60_7c44_4ad5_9fac_8101d7d0c3ac.slice/crio-1a172d3b0b33cc78dd9f35b85b61d873e89b2debbe1cf6716d6f2733a716e323 WatchSource:0}: Error finding container 1a172d3b0b33cc78dd9f35b85b61d873e89b2debbe1cf6716d6f2733a716e323: Status 404 returned error can't find the container with id 1a172d3b0b33cc78dd9f35b85b61d873e89b2debbe1cf6716d6f2733a716e323
Jul 27 17:32:39.038901 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.038873    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-pwdzm"]
Jul 27 17:32:39.042856 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:39.042831    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d2f48c_0254_434b_a3bf_455dd45f270a.slice/crio-bbbff4691c1ec0838c80a0044716bbbabd5ad8cf126164997571727ad0d5d4d6 WatchSource:0}: Error finding container bbbff4691c1ec0838c80a0044716bbbabd5ad8cf126164997571727ad0d5d4d6: Status 404 returned error can't find the container with id bbbff4691c1ec0838c80a0044716bbbabd5ad8cf126164997571727ad0d5d4d6
Jul 27 17:32:39.123727 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.123689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:39.123727 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.123733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:39.123920 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.123789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:39.123920 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.123807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:39.123920 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123865    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:40.123841543 +0000 UTC m=+228.202273948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : configmap references non-existent config key: service-ca.crt
Jul 27 17:32:39.123920 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123895    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:39.123920 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123900    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:32:39.124089 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123900    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:32:39.124089 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123940    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls podName:024b9af0-76c5-41b8-87d7-a763b6f1a919 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:40.123928783 +0000 UTC m=+228.202361186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-4pw8b" (UID: "024b9af0-76c5-41b8-87d7-a763b6f1a919") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:39.124089 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123953    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls podName:0b94f98e-f593-4af6-8200-98ee01580a48 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:40.123947623 +0000 UTC m=+228.202380023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls") pod "cluster-samples-operator-6748489656-tr78b" (UID: "0b94f98e-f593-4af6-8200-98ee01580a48") : secret "samples-operator-tls" not found
Jul 27 17:32:39.124089 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:39.123964    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:40.123958398 +0000 UTC m=+228.202390799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : secret "router-metrics-certs-default" not found
Jul 27 17:32:39.128586 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.128562    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" event={"ID":"79d2f48c-0254-434b-a3bf-455dd45f270a","Type":"ContainerStarted","Data":"bbbff4691c1ec0838c80a0044716bbbabd5ad8cf126164997571727ad0d5d4d6"}
Jul 27 17:32:39.129693 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.129664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jvqln" event={"ID":"ab6797ac-2bcb-45b5-94a0-6359caeff863","Type":"ContainerStarted","Data":"19603db53bdeed29ca4d35939f5271b48b1d5f923bf8635206bed7cb347a29ae"}
Jul 27 17:32:39.130680 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.130660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd" event={"ID":"3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4","Type":"ContainerStarted","Data":"1aab560bc756321e8f9de56f78f93180166b20e4da46fb1868f89c530da81167"}
Jul 27 17:32:39.131571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:39.131552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q" event={"ID":"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac","Type":"ContainerStarted","Data":"1a172d3b0b33cc78dd9f35b85b61d873e89b2debbe1cf6716d6f2733a716e323"}
Jul 27 17:32:40.132980 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:40.132942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:40.132991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:40.133035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:40.133069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133105    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133166    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:42.133146054 +0000 UTC m=+230.211578457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : secret "router-metrics-certs-default" not found
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133184    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133193    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133223    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:42.133202889 +0000 UTC m=+230.211635291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : configmap references non-existent config key: service-ca.crt
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133252    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls podName:024b9af0-76c5-41b8-87d7-a763b6f1a919 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:42.133236524 +0000 UTC m=+230.211668930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-4pw8b" (UID: "024b9af0-76c5-41b8-87d7-a763b6f1a919") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:40.133545 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:40.133269    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls podName:0b94f98e-f593-4af6-8200-98ee01580a48 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:42.133260556 +0000 UTC m=+230.211692962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls") pod "cluster-samples-operator-6748489656-tr78b" (UID: "0b94f98e-f593-4af6-8200-98ee01580a48") : secret "samples-operator-tls" not found
Jul 27 17:32:41.140354 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:41.140317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd" event={"ID":"3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4","Type":"ContainerStarted","Data":"3aef3ece1abbb77923d80782fda7fbbcccc5c4f15c73cd8dc5aba461130c5627"}
Jul 27 17:32:41.155692 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:41.155618    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-mdrqd" podStartSLOduration=1.874539111 podStartE2EDuration="3.155596882s" totalImagesPullingTime="1.281057771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:41.154610802 +0000 UTC m=+229.233043225" watchObservedRunningTime="2026-07-27 17:32:41.155596882 +0000 UTC m=+229.234029306"
Jul 27 17:32:41.226622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:41.226585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:41.279180 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:41.279145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:42.146439 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.146386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q" event={"ID":"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac","Type":"ContainerStarted","Data":"ee2ed877d0f4be38a31511c6b0c7239ea4a005af0d30db890c62f219d337e663"}
Jul 27 17:32:42.148818 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.148790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jvqln" event={"ID":"ab6797ac-2bcb-45b5-94a0-6359caeff863","Type":"ContainerStarted","Data":"8eb87d8cb365227d0590a8af54d1334274b98b50b4109ee15a4669f39591056d"}
Jul 27 17:32:42.152104 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.152083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:42.152239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.152120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:42.152239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.152183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:42.152239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.152217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:42.152397 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152270    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:32:42.152397 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152335    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:46.152315919 +0000 UTC m=+234.230748326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : secret "router-metrics-certs-default" not found
Jul 27 17:32:42.152397 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152380    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:32:42.152590 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152446    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls podName:0b94f98e-f593-4af6-8200-98ee01580a48 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:46.152414171 +0000 UTC m=+234.230846574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls") pod "cluster-samples-operator-6748489656-tr78b" (UID: "0b94f98e-f593-4af6-8200-98ee01580a48") : secret "samples-operator-tls" not found
Jul 27 17:32:42.152590 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152505    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:42.152590 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152524    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:46.152511837 +0000 UTC m=+234.230944242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : configmap references non-existent config key: service-ca.crt
Jul 27 17:32:42.152590 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.152553    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls podName:024b9af0-76c5-41b8-87d7-a763b6f1a919 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:46.152539819 +0000 UTC m=+234.230972229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-4pw8b" (UID: "024b9af0-76c5-41b8-87d7-a763b6f1a919") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:42.162077 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.162030    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q" podStartSLOduration=1.733649771 podStartE2EDuration="4.162012176s" totalImagesPullingTime="2.428362405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:42.16120129 +0000 UTC m=+230.239633713" watchObservedRunningTime="2026-07-27 17:32:42.162012176 +0000 UTC m=+230.240444601"
Jul 27 17:32:42.178532 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.178478    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-jvqln" podStartSLOduration=1.688203235 podStartE2EDuration="4.178458505s" totalImagesPullingTime="2.49025527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:42.178255687 +0000 UTC m=+230.256688111" watchObservedRunningTime="2026-07-27 17:32:42.178458505 +0000 UTC m=+230.256890927"
Jul 27 17:32:42.607375 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.607343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:42.656864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.656824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:42.738073 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.738029    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9jp2v"]
Jul 27 17:32:42.740420 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.740394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.743301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.743269    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 17:32:42.743451 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.743365    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8r9mw\""
Jul 27 17:32:42.743451 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.743404    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 17:32:42.754800 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.752894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9jp2v"]
Jul 27 17:32:42.762669 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.762577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:42.857202 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.857162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7275deb5-c992-47c5-91fb-eb478fba9a30-data-volume\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.857363 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.857229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q9bg\" (UniqueName: \"kubernetes.io/projected/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-api-access-2q9bg\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.857363 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.857269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7275deb5-c992-47c5-91fb-eb478fba9a30-crio-socket\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.857363 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.857326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.857594 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.857369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.884160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.884108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:42.958626 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.958589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.958820 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.958654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7275deb5-c992-47c5-91fb-eb478fba9a30-data-volume\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.958820 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.958744    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:32:42.958820 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.958801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9bg\" (UniqueName: \"kubernetes.io/projected/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-api-access-2q9bg\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.959007 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.958819    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:43.458797925 +0000 UTC m=+231.537230330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : configmap "kube-rbac-proxy" not found
Jul 27 17:32:42.959007 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.958918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7275deb5-c992-47c5-91fb-eb478fba9a30-crio-socket\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.959007 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.958992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.959196 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.959025    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7275deb5-c992-47c5-91fb-eb478fba9a30-crio-socket\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.959196 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.959084    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7275deb5-c992-47c5-91fb-eb478fba9a30-data-volume\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:42.959196 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.959097    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:42.959196 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:42.959142    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:43.459127064 +0000 UTC m=+231.537559493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:42.967370 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:42.967335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9bg\" (UniqueName: \"kubernetes.io/projected/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-api-access-2q9bg\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:43.085531 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:43.085487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:43.186040 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:43.185964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:43.318818 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:43.316742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:43.416770 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:43.416744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:43.463324 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:43.463292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:43.463499 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:43.463356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:43.463499 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:43.463454    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:43.463499 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:43.463480    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:32:43.463652 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:43.463521    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:44.463504945 +0000 UTC m=+232.541937346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:43.463652 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:43.463536    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:44.463529042 +0000 UTC m=+232.541961444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : configmap "kube-rbac-proxy" not found
Jul 27 17:32:44.090344 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.090316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:44.156905 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.156878    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/0.log"
Jul 27 17:32:44.157058 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.156917    2577 generic.go:356] "Generic (PLEG): container finished" podID="79d2f48c-0254-434b-a3bf-455dd45f270a" containerID="287551e486e8e6a77a876f99f00763ddab32f43d6b50d33e75daa599bc9f8689" exitCode=255
Jul 27 17:32:44.157058 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.156972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" event={"ID":"79d2f48c-0254-434b-a3bf-455dd45f270a","Type":"ContainerDied","Data":"287551e486e8e6a77a876f99f00763ddab32f43d6b50d33e75daa599bc9f8689"}
Jul 27 17:32:44.157251 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.157238    2577 scope.go:117] "RemoveContainer" containerID="287551e486e8e6a77a876f99f00763ddab32f43d6b50d33e75daa599bc9f8689"
Jul 27 17:32:44.177022 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.176994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:44.231831 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.231803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:44.472715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.472684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:44.472874 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.472740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:44.472874 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:44.472833    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:44.472948 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:44.472894    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:46.472880209 +0000 UTC m=+234.551312613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:44.472948 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:44.472833    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:32:44.473017 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:44.472963    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:46.472952398 +0000 UTC m=+234.551384802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : configmap "kube-rbac-proxy" not found
Jul 27 17:32:44.912501 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.912418    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-q2vvq"]
Jul 27 17:32:44.912637 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.912505    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w84kh_1cfaef06-f66e-4e0d-9cae-01a4f177dd39/dns-node-resolver/0.log"
Jul 27 17:32:44.914690 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.914671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:44.917285 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.917265    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 27 17:32:44.917941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.917926    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 27 17:32:44.918572 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.918554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r4zr7\""
Jul 27 17:32:44.918659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.918610    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 27 17:32:44.918659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.918614    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 27 17:32:44.923111 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.923094    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-q2vvq"]
Jul 27 17:32:44.949057 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.949026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:44.977454 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.977415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/93de8f22-6b34-41fe-acb2-0a899c1f330a-signing-cabundle\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:44.977544 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.977458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd54d\" (UniqueName: \"kubernetes.io/projected/93de8f22-6b34-41fe-acb2-0a899c1f330a-kube-api-access-rd54d\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:44.977544 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:44.977503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/93de8f22-6b34-41fe-acb2-0a899c1f330a-signing-key\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.000416 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.000385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:45.078860 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.078829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/93de8f22-6b34-41fe-acb2-0a899c1f330a-signing-cabundle\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.078860 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.078860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd54d\" (UniqueName: \"kubernetes.io/projected/93de8f22-6b34-41fe-acb2-0a899c1f330a-kube-api-access-rd54d\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.079138 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.078888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/93de8f22-6b34-41fe-acb2-0a899c1f330a-signing-key\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.079561 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.079541    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/93de8f22-6b34-41fe-acb2-0a899c1f330a-signing-cabundle\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.081253 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.081236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/93de8f22-6b34-41fe-acb2-0a899c1f330a-signing-key\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.087295 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.087276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd54d\" (UniqueName: \"kubernetes.io/projected/93de8f22-6b34-41fe-acb2-0a899c1f330a-kube-api-access-rd54d\") pod \"service-ca-658d7b7c78-q2vvq\" (UID: \"93de8f22-6b34-41fe-acb2-0a899c1f330a\") " pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.162135 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.162106    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:32:45.162549 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.162507    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/0.log"
Jul 27 17:32:45.162628 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.162547    2577 generic.go:356] "Generic (PLEG): container finished" podID="79d2f48c-0254-434b-a3bf-455dd45f270a" containerID="20bd697f4d1436190cfed99f0d42ec6966d1f131ade9f5d52cb8042896073243" exitCode=255
Jul 27 17:32:45.162628 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.162580    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" event={"ID":"79d2f48c-0254-434b-a3bf-455dd45f270a","Type":"ContainerDied","Data":"20bd697f4d1436190cfed99f0d42ec6966d1f131ade9f5d52cb8042896073243"}
Jul 27 17:32:45.162628 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.162619    2577 scope.go:117] "RemoveContainer" containerID="287551e486e8e6a77a876f99f00763ddab32f43d6b50d33e75daa599bc9f8689"
Jul 27 17:32:45.162865 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.162851    2577 scope.go:117] "RemoveContainer" containerID="20bd697f4d1436190cfed99f0d42ec6966d1f131ade9f5d52cb8042896073243"
Jul 27 17:32:45.163033 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:45.163016    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-pwdzm_openshift-console-operator(79d2f48c-0254-434b-a3bf-455dd45f270a)\"" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" podUID="79d2f48c-0254-434b-a3bf-455dd45f270a"
Jul 27 17:32:45.223376 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.223346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq"
Jul 27 17:32:45.342095 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.342053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-q2vvq"]
Jul 27 17:32:45.347665 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:45.347521    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93de8f22_6b34_41fe_acb2_0a899c1f330a.slice/crio-a62d1622e7a21b5568ebf7888f5f50aa60f3644e8242d850ac541848acfd2d64 WatchSource:0}: Error finding container a62d1622e7a21b5568ebf7888f5f50aa60f3644e8242d850ac541848acfd2d64: Status 404 returned error can't find the container with id a62d1622e7a21b5568ebf7888f5f50aa60f3644e8242d850ac541848acfd2d64
Jul 27 17:32:45.506167 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.506136    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-72mk7_75d73dec-5e9b-48a8-882b-4f67668fb911/node-ca/0.log"
Jul 27 17:32:45.975622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:45.975591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:46.168067 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.168040    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:32:46.168471 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.168452    2577 scope.go:117] "RemoveContainer" containerID="20bd697f4d1436190cfed99f0d42ec6966d1f131ade9f5d52cb8042896073243"
Jul 27 17:32:46.168672 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.168643    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-pwdzm_openshift-console-operator(79d2f48c-0254-434b-a3bf-455dd45f270a)\"" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" podUID="79d2f48c-0254-434b-a3bf-455dd45f270a"
Jul 27 17:32:46.169634 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.169611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq" event={"ID":"93de8f22-6b34-41fe-acb2-0a899c1f330a","Type":"ContainerStarted","Data":"3bceeeef0c35c13b2ddc2cf1155181f337723c653db936f5929461f96f718eae"}
Jul 27 17:32:46.169717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.169653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq" event={"ID":"93de8f22-6b34-41fe-acb2-0a899c1f330a","Type":"ContainerStarted","Data":"a62d1622e7a21b5568ebf7888f5f50aa60f3644e8242d850ac541848acfd2d64"}
Jul 27 17:32:46.188733 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.188706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:46.188858 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.188748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:46.188858 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.188797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:46.188858 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.188841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188860    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188911    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188924    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:54.188905837 +0000 UTC m=+242.267338239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : secret "router-metrics-certs-default" not found
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188939    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle podName:7c2bc8d6-6d7a-4453-8c15-129def8d3bea nodeName:}" failed. No retries permitted until 2026-07-27 17:32:54.188932622 +0000 UTC m=+242.267365022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle") pod "router-default-b46967c9f-ptz2d" (UID: "7c2bc8d6-6d7a-4453-8c15-129def8d3bea") : configmap references non-existent config key: service-ca.crt
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188959    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188969    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls podName:024b9af0-76c5-41b8-87d7-a763b6f1a919 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:54.188951986 +0000 UTC m=+242.267384395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-4pw8b" (UID: "024b9af0-76c5-41b8-87d7-a763b6f1a919") : secret "cluster-monitoring-operator-tls" not found
Jul 27 17:32:46.189013 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.188999    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls podName:0b94f98e-f593-4af6-8200-98ee01580a48 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:54.188986858 +0000 UTC m=+242.267419262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls") pod "cluster-samples-operator-6748489656-tr78b" (UID: "0b94f98e-f593-4af6-8200-98ee01580a48") : secret "samples-operator-tls" not found
Jul 27 17:32:46.202032 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.201984    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-q2vvq" podStartSLOduration=2.201968212 podStartE2EDuration="2.201968212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:46.200805507 +0000 UTC m=+234.279237931" watchObservedRunningTime="2026-07-27 17:32:46.201968212 +0000 UTC m=+234.280400634"
Jul 27 17:32:46.306417 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.306347    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmr84_32b34b39-946f-4c7b-855b-7905b1b84903/serve-healthcheck-canary/0.log"
Jul 27 17:32:46.492284 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.492246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:46.492768 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.492298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:46.492768 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.492406    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:46.492768 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.492442    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:32:46.492768 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.492486    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:50.492470167 +0000 UTC m=+238.570902569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:46.492768 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:46.492502    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:50.492495162 +0000 UTC m=+238.570927563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : configmap "kube-rbac-proxy" not found
Jul 27 17:32:46.776756 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:46.776709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:48.388658 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.388609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:48.488936 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.488908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:48.674227 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.674146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:48.884134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.884105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:48.893060 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.893036    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:48.893177 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.893096    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:32:48.893515 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:48.893502    2577 scope.go:117] "RemoveContainer" containerID="20bd697f4d1436190cfed99f0d42ec6966d1f131ade9f5d52cb8042896073243"
Jul 27 17:32:48.893695 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:48.893675    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-pwdzm_openshift-console-operator(79d2f48c-0254-434b-a3bf-455dd45f270a)\"" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" podUID="79d2f48c-0254-434b-a3bf-455dd45f270a"
Jul 27 17:32:49.078156 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:49.078127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:49.180100 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:49.180066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:49.280217 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:49.280183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:49.479128 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:49.479100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:49.698212 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:49.698183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:50.527243 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:50.527211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:50.527763 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:50.527275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:50.527763 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:50.527368    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:50.527763 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:50.527418    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 17:32:50.527763 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:50.527451    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:58.527418277 +0000 UTC m=+246.605850677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : secret "insights-runtime-extractor-tls" not found
Jul 27 17:32:50.527763 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:32:50.527497    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm podName:7275deb5-c992-47c5-91fb-eb478fba9a30 nodeName:}" failed. No retries permitted until 2026-07-27 17:32:58.527482001 +0000 UTC m=+246.605914406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9jp2v" (UID: "7275deb5-c992-47c5-91fb-eb478fba9a30") : configmap "kube-rbac-proxy" not found
Jul 27 17:32:50.551961 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:50.551933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:52.199174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:52.199132    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:52.300561 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:52.300532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:52.400759 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:52.400731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:52.678192 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:52.678163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:53.070538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:53.070501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:53.171560 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:53.171530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:53.271912 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:53.271877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:53.668350 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:53.668319    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:54.261327 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.261294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:54.261327 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.261330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:54.261608 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.261363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:54.261608 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.261403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:54.262076 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.262052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-service-ca-bundle\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:54.263727 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.263708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/024b9af0-76c5-41b8-87d7-a763b6f1a919-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-4pw8b\" (UID: \"024b9af0-76c5-41b8-87d7-a763b6f1a919\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:54.263888 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.263868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2bc8d6-6d7a-4453-8c15-129def8d3bea-metrics-certs\") pod \"router-default-b46967c9f-ptz2d\" (UID: \"7c2bc8d6-6d7a-4453-8c15-129def8d3bea\") " pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:54.263946 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.263918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b94f98e-f593-4af6-8200-98ee01580a48-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-tr78b\" (UID: \"0b94f98e-f593-4af6-8200-98ee01580a48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:54.267258 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.267240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:54.390840 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.390815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wcwl9\""
Jul 27 17:32:54.398817 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.398800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"
Jul 27 17:32:54.415496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.415476    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kq7kc\""
Jul 27 17:32:54.421493 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.421473    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-q5f5c\""
Jul 27 17:32:54.423329 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.423305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"
Jul 27 17:32:54.429502 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.429456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:54.544998 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.544962    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b"]
Jul 27 17:32:54.548368 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:54.548339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024b9af0_76c5_41b8_87d7_a763b6f1a919.slice/crio-0a285376eaebf8c17c1cb98b750fd5867715d948a5e8ea78edec0230828074c5 WatchSource:0}: Error finding container 0a285376eaebf8c17c1cb98b750fd5867715d948a5e8ea78edec0230828074c5: Status 404 returned error can't find the container with id 0a285376eaebf8c17c1cb98b750fd5867715d948a5e8ea78edec0230828074c5
Jul 27 17:32:54.562577 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.562523    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b"]
Jul 27 17:32:54.585865 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.585794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b46967c9f-ptz2d"]
Jul 27 17:32:54.590093 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:54.590069    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2bc8d6_6d7a_4453_8c15_129def8d3bea.slice/crio-bbc668afa5eb6c619acfc6989feec066e0f8318b9ffee100b4fb41c2bed82c55 WatchSource:0}: Error finding container bbc668afa5eb6c619acfc6989feec066e0f8318b9ffee100b4fb41c2bed82c55: Status 404 returned error can't find the container with id bbc668afa5eb6c619acfc6989feec066e0f8318b9ffee100b4fb41c2bed82c55
Jul 27 17:32:54.871188 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:54.871110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:55.072331 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.072278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:55.198979 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.198903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b" event={"ID":"0b94f98e-f593-4af6-8200-98ee01580a48","Type":"ContainerStarted","Data":"3876388cf301590b03b988645c5d8fd04a41ecf5256f6e20dbdfced23de315d3"}
Jul 27 17:32:55.200400 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.200376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b46967c9f-ptz2d" event={"ID":"7c2bc8d6-6d7a-4453-8c15-129def8d3bea","Type":"ContainerStarted","Data":"a91f94270678b01dc343411d3652fafa1fc16ae5b27198c9dd0870d3f35c3cc9"}
Jul 27 17:32:55.200557 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.200416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b46967c9f-ptz2d" event={"ID":"7c2bc8d6-6d7a-4453-8c15-129def8d3bea","Type":"ContainerStarted","Data":"bbc668afa5eb6c619acfc6989feec066e0f8318b9ffee100b4fb41c2bed82c55"}
Jul 27 17:32:55.201475 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.201454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b" event={"ID":"024b9af0-76c5-41b8-87d7-a763b6f1a919","Type":"ContainerStarted","Data":"0a285376eaebf8c17c1cb98b750fd5867715d948a5e8ea78edec0230828074c5"}
Jul 27 17:32:55.220189 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.220150    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-b46967c9f-ptz2d" podStartSLOduration=17.220134854 podStartE2EDuration="17.220134854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:55.218161839 +0000 UTC m=+243.296594262" watchObservedRunningTime="2026-07-27 17:32:55.220134854 +0000 UTC m=+243.298567280"
Jul 27 17:32:55.429717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.429681    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:55.432298 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.432269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:55.472614 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.472586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:55.696815 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:55.696778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:56.000617 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:56.000573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:56.101854 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:56.101819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:56.205383 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:56.205351    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:56.206660 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:56.206638    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b46967c9f-ptz2d"
Jul 27 17:32:56.479303 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:56.479269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:57.076831 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.076705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:57.211227 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.211184    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b" event={"ID":"0b94f98e-f593-4af6-8200-98ee01580a48","Type":"ContainerStarted","Data":"fe6cb8a526cc003eedbaae60de0004f231bb543eff2a20e34a5c6f66a4f75ff6"}
Jul 27 17:32:57.211400 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.211255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b" event={"ID":"0b94f98e-f593-4af6-8200-98ee01580a48","Type":"ContainerStarted","Data":"006bc8a76e1343ec07fac539d9677a5f7bd23a6a70214cacea91e2076d602f72"}
Jul 27 17:32:57.230873 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.230818    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-tr78b" podStartSLOduration=17.093152613 podStartE2EDuration="19.230799398s" totalImagesPullingTime="2.137646785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:57.229193001 +0000 UTC m=+245.307625425" watchObservedRunningTime="2026-07-27 17:32:57.230799398 +0000 UTC m=+245.309231825"
Jul 27 17:32:57.275256 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.275227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:57.475119 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.475089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:57.697231 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:57.697197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:58.219310 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.219277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b" event={"ID":"024b9af0-76c5-41b8-87d7-a763b6f1a919","Type":"ContainerStarted","Data":"bcee77fcf0d7a8cc44f3f3cc49dc038b6447591cea3f09842c32792656fc66d7"}
Jul 27 17:32:58.237901 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.237842    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b" podStartSLOduration=16.72558232 podStartE2EDuration="20.237819688s" totalImagesPullingTime="3.512237368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:32:58.23722852 +0000 UTC m=+246.315660956" watchObservedRunningTime="2026-07-27 17:32:58.237819688 +0000 UTC m=+246.316252112"
Jul 27 17:32:58.359033 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.358957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:58.461350 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.461318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:58.558654 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.558612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:58.599948 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.599910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:58.600137 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.599955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:58.600462 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.600420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7275deb5-c992-47c5-91fb-eb478fba9a30-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:58.602277 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.602254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7275deb5-c992-47c5-91fb-eb478fba9a30-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9jp2v\" (UID: \"7275deb5-c992-47c5-91fb-eb478fba9a30\") " pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:58.653388 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.653326    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8r9mw\""
Jul 27 17:32:58.658805 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.658778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:32:58.660677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.660663    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9jp2v"
Jul 27 17:32:58.781258 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:58.781222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9jp2v"]
Jul 27 17:32:58.784160 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:32:58.784134    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7275deb5_c992_47c5_91fb_eb478fba9a30.slice/crio-7509bf03f7681417b933a11c4d88e57515a1be4e16485d84c6b4efb6e62de4d1 WatchSource:0}: Error finding container 7509bf03f7681417b933a11c4d88e57515a1be4e16485d84c6b4efb6e62de4d1: Status 404 returned error can't find the container with id 7509bf03f7681417b933a11c4d88e57515a1be4e16485d84c6b4efb6e62de4d1
Jul 27 17:32:59.223413 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:32:59.223382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9jp2v" event={"ID":"7275deb5-c992-47c5-91fb-eb478fba9a30","Type":"ContainerStarted","Data":"7509bf03f7681417b933a11c4d88e57515a1be4e16485d84c6b4efb6e62de4d1"}
Jul 27 17:33:03.238375 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.238336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:33:03.240825 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.240801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7d5444a-98d9-4fe7-8017-a35216c24332-metrics-certs\") pod \"network-metrics-daemon-29lnk\" (UID: \"c7d5444a-98d9-4fe7-8017-a35216c24332\") " pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:33:03.357359 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.357328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:03.380418 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.380390    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qwvcb\""
Jul 27 17:33:03.388570 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.388539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29lnk"
Jul 27 17:33:03.409559 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.409527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:03.821014 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:03.820989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29lnk"]
Jul 27 17:33:03.824119 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:03.824096    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d5444a_98d9_4fe7_8017_a35216c24332.slice/crio-3fd288dac4a54483f6dbbb18039a2ae5d506df2451e091a8522fb0754121cf6a WatchSource:0}: Error finding container 3fd288dac4a54483f6dbbb18039a2ae5d506df2451e091a8522fb0754121cf6a: Status 404 returned error can't find the container with id 3fd288dac4a54483f6dbbb18039a2ae5d506df2451e091a8522fb0754121cf6a
Jul 27 17:33:04.243246 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:04.243208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29lnk" event={"ID":"c7d5444a-98d9-4fe7-8017-a35216c24332","Type":"ContainerStarted","Data":"3fd288dac4a54483f6dbbb18039a2ae5d506df2451e091a8522fb0754121cf6a"}
Jul 27 17:33:04.244538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:04.244513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9jp2v" event={"ID":"7275deb5-c992-47c5-91fb-eb478fba9a30","Type":"ContainerStarted","Data":"bfaccdfbd928fafd8982886e2fa1402d1482fe19ddfae65f16c845e85b6ea80d"}
Jul 27 17:33:04.477699 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:04.477656    2577 scope.go:117] "RemoveContainer" containerID="20bd697f4d1436190cfed99f0d42ec6966d1f131ade9f5d52cb8042896073243"
Jul 27 17:33:05.250115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:05.250080    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:33:05.250673 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:05.250145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" event={"ID":"79d2f48c-0254-434b-a3bf-455dd45f270a","Type":"ContainerStarted","Data":"d210f2a8729af8585f4b166a87440fc3975f7dbd822ccab980774a123450d6c6"}
Jul 27 17:33:05.250673 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:05.250449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:33:05.266445 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:05.266380    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm" podStartSLOduration=23.001760673 podStartE2EDuration="27.26636203s" totalImagesPullingTime="4.264601357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:05.265850498 +0000 UTC m=+253.344282923" watchObservedRunningTime="2026-07-27 17:33:05.26636203 +0000 UTC m=+253.344794454"
Jul 27 17:33:05.833000 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:05.832969    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-pwdzm"
Jul 27 17:33:06.043475 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.043404    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.143173 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.143092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.243659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.243631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.343614 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.343535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.494343 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.494297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.589278 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.589245    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"]
Jul 27 17:33:06.590731 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.590691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.609387 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.608300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"]
Jul 27 17:33:06.609387 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.608437    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:06.613139 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.613114    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sv5sz\""
Jul 27 17:33:06.613268 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.613137    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 27 17:33:06.673836 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.673804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.692969 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.692938    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"]
Jul 27 17:33:06.698784 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.698759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.701520 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.701496    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-skq5m\""
Jul 27 17:33:06.701640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.701524    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 17:33:06.701640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.701541    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 17:33:06.701640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.701524    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 17:33:06.706230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.706207    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 17:33:06.708665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.708642    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"]
Jul 27 17:33:06.770090 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a07f34b7-b180-4541-af7a-84217e5276b2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jcnth\" (UID: \"a07f34b7-b180-4541-af7a-84217e5276b2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:06.770239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-image-registry-private-configuration\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-registry-certificates\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-installation-pull-secrets\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770353 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhgkv\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-kube-api-access-vhgkv\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770353 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.770353 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770278    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-registry-tls\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770465 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-bound-sa-token\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770465 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-trusted-ca\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.770465 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.770422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-ca-trust-extracted\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.871778 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-registry-tls\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.871953 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-bound-sa-token\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.871953 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-trusted-ca\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.871953 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-ca-trust-extracted\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.871953 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a07f34b7-b180-4541-af7a-84217e5276b2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jcnth\" (UID: \"a07f34b7-b180-4541-af7a-84217e5276b2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:06.872211 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-image-registry-private-configuration\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.872211 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.871987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-registry-certificates\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.872211 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.872014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-installation-pull-secrets\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.872211 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.872043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhgkv\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-kube-api-access-vhgkv\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.873071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.873045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-registry-certificates\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.873870 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.873846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-trusted-ca\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.874046 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.874020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-ca-trust-extracted\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.875873 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.875852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a07f34b7-b180-4541-af7a-84217e5276b2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jcnth\" (UID: \"a07f34b7-b180-4541-af7a-84217e5276b2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:06.876039 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.876019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-image-registry-private-configuration\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.876506 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.876483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-registry-tls\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.876701 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.876658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-installation-pull-secrets\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.886784 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.886757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhgkv\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-kube-api-access-vhgkv\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.888081 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.888050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4d74892-4cd7-451c-9b51-6d96d48f3fa7-bound-sa-token\") pod \"image-registry-d8bcdfb7f-gtf7k\" (UID: \"c4d74892-4cd7-451c-9b51-6d96d48f3fa7\") " pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:06.918399 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.918352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:06.920402 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.920375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:06.996558 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:06.996520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:07.008564 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:07.008528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:07.097793 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:07.097714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:07.495309 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:07.495160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:07.579570 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:07.579539    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"]
Jul 27 17:33:07.607259 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:07.607228    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"]
Jul 27 17:33:07.695231 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:07.695144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:07.708470 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:07.708418    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07f34b7_b180_4541_af7a_84217e5276b2.slice/crio-7f9c025de394b9a2351ed1a6c97a329d07f4e473fe6397401d6d3e9063b0a4cd WatchSource:0}: Error finding container 7f9c025de394b9a2351ed1a6c97a329d07f4e473fe6397401d6d3e9063b0a4cd: Status 404 returned error can't find the container with id 7f9c025de394b9a2351ed1a6c97a329d07f4e473fe6397401d6d3e9063b0a4cd
Jul 27 17:33:07.709097 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:07.709073    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d74892_4cd7_451c_9b51_6d96d48f3fa7.slice/crio-3298e3048a570f161744ab15a8ce8aa5c574fe53950bd2bbf23e27e12629773b WatchSource:0}: Error finding container 3298e3048a570f161744ab15a8ce8aa5c574fe53950bd2bbf23e27e12629773b: Status 404 returned error can't find the container with id 3298e3048a570f161744ab15a8ce8aa5c574fe53950bd2bbf23e27e12629773b
Jul 27 17:33:08.139291 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.139260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:08.262474 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.262421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k" event={"ID":"c4d74892-4cd7-451c-9b51-6d96d48f3fa7","Type":"ContainerStarted","Data":"67e29c4aff0a76f4987af19798b7355646beacfb8ced992146f392abe5e72ff6"}
Jul 27 17:33:08.262664 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.262494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k" event={"ID":"c4d74892-4cd7-451c-9b51-6d96d48f3fa7","Type":"ContainerStarted","Data":"3298e3048a570f161744ab15a8ce8aa5c574fe53950bd2bbf23e27e12629773b"}
Jul 27 17:33:08.262664 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.262549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:08.263684 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.263655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth" event={"ID":"a07f34b7-b180-4541-af7a-84217e5276b2","Type":"ContainerStarted","Data":"7f9c025de394b9a2351ed1a6c97a329d07f4e473fe6397401d6d3e9063b0a4cd"}
Jul 27 17:33:08.265332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.265303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29lnk" event={"ID":"c7d5444a-98d9-4fe7-8017-a35216c24332","Type":"ContainerStarted","Data":"e81db932647faac740d475c4cc4195c96c40a0c726cb74b6ec82c3574c303bdc"}
Jul 27 17:33:08.265481 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.265350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29lnk" event={"ID":"c7d5444a-98d9-4fe7-8017-a35216c24332","Type":"ContainerStarted","Data":"815a0bd9cf1e12401fa045deb3f2004e9a2a43eac3ea3b3c908c1a2a5f038c56"}
Jul 27 17:33:08.266931 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.266912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9jp2v" event={"ID":"7275deb5-c992-47c5-91fb-eb478fba9a30","Type":"ContainerStarted","Data":"15e3d7efea22de5e6eca852d55fc1b5aa1b3e1abb99acf10a52fbe3b8e0834f9"}
Jul 27 17:33:08.280529 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.280483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k" podStartSLOduration=2.280466931 podStartE2EDuration="2.280466931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:08.279564902 +0000 UTC m=+256.357997327" watchObservedRunningTime="2026-07-27 17:33:08.280466931 +0000 UTC m=+256.358899355"
Jul 27 17:33:08.298141 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.298093    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29lnk" podStartSLOduration=252.384434957 podStartE2EDuration="4m16.298075004s" totalImagesPullingTime="3.913640047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:08.296648785 +0000 UTC m=+256.375081211" watchObservedRunningTime="2026-07-27 17:33:08.298075004 +0000 UTC m=+256.376507430"
Jul 27 17:33:08.396622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.396527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:08.497667 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.497612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:08.548868 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:08.548830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:09.272691 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:09.272632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth" event={"ID":"a07f34b7-b180-4541-af7a-84217e5276b2","Type":"ContainerStarted","Data":"04cd82ec9cb8a422ae506ae58345e9597762914230e32cec09e204eb63749219"}
Jul 27 17:33:09.272857 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:09.272724    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:09.278314 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:09.278286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth"
Jul 27 17:33:09.289290 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:09.289243    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jcnth" podStartSLOduration=2.05068054 podStartE2EDuration="3.289228898s" totalImagesPullingTime="1.238548358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:09.286771376 +0000 UTC m=+257.365203798" watchObservedRunningTime="2026-07-27 17:33:09.289228898 +0000 UTC m=+257.367661338"
Jul 27 17:33:09.592938 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:09.592859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:09.652098 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:09.652064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:12.286495 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:12.286460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9jp2v" event={"ID":"7275deb5-c992-47c5-91fb-eb478fba9a30","Type":"ContainerStarted","Data":"4bfea09bf4f0f682bbc4e1769a04c9a0a4bb7ba0d4d387eb32cdd24f308df960"}
Jul 27 17:33:12.908504 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:12.908473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:13.112999 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.112941    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9jp2v" podStartSLOduration=18.659189522 podStartE2EDuration="31.112918926s" totalImagesPullingTime="12.453729404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:32:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:12.303178281 +0000 UTC m=+260.381610738" watchObservedRunningTime="2026-07-27 17:33:13.112918926 +0000 UTC m=+261.191351348"
Jul 27 17:33:13.114134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.114110    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54595664d9-flhcq"]
Jul 27 17:33:13.116937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.116920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.121934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.120004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 17:33:13.121934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.120289    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 17:33:13.121934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.121391    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 17:33:13.121934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.121686    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 17:33:13.121934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.121842    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 17:33:13.121934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.121868    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-stmxj\""
Jul 27 17:33:13.122241 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.122011    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 17:33:13.122241 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.122040    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 17:33:13.126382 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.126361    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 17:33:13.126525 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.126505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:13.126880 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.126856    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54595664d9-flhcq"]
Jul 27 17:33:13.180322 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.180239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:13.226837 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.226811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-serving-cert\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.226932 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.226845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-service-ca\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.226932 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.226871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-trusted-ca-bundle\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.226932 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.226926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-oauth-serving-cert\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.227070 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.226945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-console-config\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.227070 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.227008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-oauth-config\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.227070 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.227038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzz82\" (UniqueName: \"kubernetes.io/projected/6964f47f-8c92-48e8-be56-03868e248815-kube-api-access-mzz82\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.327687 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.327650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-serving-cert\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.327698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-service-ca\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.327733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-trusted-ca-bundle\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.327941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-oauth-serving-cert\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.327979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-console-config\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.328027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-oauth-config\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.328044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzz82\" (UniqueName: \"kubernetes.io/projected/6964f47f-8c92-48e8-be56-03868e248815-kube-api-access-mzz82\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328691 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.328663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-service-ca\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328765 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.328715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-oauth-serving-cert\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.328765 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.328755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-console-config\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.329107 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.329084    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-trusted-ca-bundle\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.330389 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.330367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-serving-cert\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.330490 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.330408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-oauth-config\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.335038 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.335018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzz82\" (UniqueName: \"kubernetes.io/projected/6964f47f-8c92-48e8-be56-03868e248815-kube-api-access-mzz82\") pod \"console-54595664d9-flhcq\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") " pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.432298 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.432211    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:13.558910 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:13.558881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54595664d9-flhcq"]
Jul 27 17:33:13.561773 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:13.561746    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6964f47f_8c92_48e8_be56_03868e248815.slice/crio-9eb30361691bd94751f95f3f7d72bd831f56dcbecc131eaedf7f3c1a20d37f6b WatchSource:0}: Error finding container 9eb30361691bd94751f95f3f7d72bd831f56dcbecc131eaedf7f3c1a20d37f6b: Status 404 returned error can't find the container with id 9eb30361691bd94751f95f3f7d72bd831f56dcbecc131eaedf7f3c1a20d37f6b
Jul 27 17:33:14.295806 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:14.295765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54595664d9-flhcq" event={"ID":"6964f47f-8c92-48e8-be56-03868e248815","Type":"ContainerStarted","Data":"9eb30361691bd94751f95f3f7d72bd831f56dcbecc131eaedf7f3c1a20d37f6b"}
Jul 27 17:33:14.664851 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:14.664821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:14.866221 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:14.866192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:15.057760 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:15.057722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:15.158331 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:15.158302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:15.301505 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:15.301479    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:33:15.301665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:15.301538    2577 generic.go:356] "Generic (PLEG): container finished" podID="024b9af0-76c5-41b8-87d7-a763b6f1a919" containerID="bcee77fcf0d7a8cc44f3f3cc49dc038b6447591cea3f09842c32792656fc66d7" exitCode=2
Jul 27 17:33:15.301665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:15.301614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b" event={"ID":"024b9af0-76c5-41b8-87d7-a763b6f1a919","Type":"ContainerDied","Data":"bcee77fcf0d7a8cc44f3f3cc49dc038b6447591cea3f09842c32792656fc66d7"}
Jul 27 17:33:15.302065 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:15.302036    2577 scope.go:117] "RemoveContainer" containerID="bcee77fcf0d7a8cc44f3f3cc49dc038b6447591cea3f09842c32792656fc66d7"
Jul 27 17:33:16.310132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:16.310095    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:33:16.310603 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:16.310224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-4pw8b" event={"ID":"024b9af0-76c5-41b8-87d7-a763b6f1a919","Type":"ContainerStarted","Data":"b9819e5e63cbfae7a4a0c87f6ad86925eab31b8303b4ee88cdd05b26c5b3f87f"}
Jul 27 17:33:16.312238 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:16.312190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54595664d9-flhcq" event={"ID":"6964f47f-8c92-48e8-be56-03868e248815","Type":"ContainerStarted","Data":"b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4"}
Jul 27 17:33:16.342146 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:16.342095    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54595664d9-flhcq" podStartSLOduration=0.677888531 podStartE2EDuration="3.342079773s" totalImagesPullingTime="2.664191242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:16.340731489 +0000 UTC m=+264.419163913" watchObservedRunningTime="2026-07-27 17:33:16.342079773 +0000 UTC m=+264.420512196"
Jul 27 17:33:19.795057 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.795022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:19.841645 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.841605    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kcv6b"]
Jul 27 17:33:19.845781 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.845751    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.848138 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.848114    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 17:33:19.848269 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.848158    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jrhwj\""
Jul 27 17:33:19.848390 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.848365    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 17:33:19.849094 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.849050    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 17:33:19.849160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.849089    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 17:33:19.884049 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-tls\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884194 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-wtmp\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884194 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f45f9ff-9b97-4b95-8834-80308aadc3a1-metrics-client-ca\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884287 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-root\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884287 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884371 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884371 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-textfile\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884371 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddtmd\" (UniqueName: \"kubernetes.io/projected/2f45f9ff-9b97-4b95-8834-80308aadc3a1-kube-api-access-ddtmd\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.884371 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.884367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-sys\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.929902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.929868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:19.984990 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.984960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-wtmp\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f45f9ff-9b97-4b95-8834-80308aadc3a1-metrics-client-ca\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-root\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-root\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-textfile\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-wtmp\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddtmd\" (UniqueName: \"kubernetes.io/projected/2f45f9ff-9b97-4b95-8834-80308aadc3a1-kube-api-access-ddtmd\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985635 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-sys\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985635 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-tls\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985635 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:33:19.985355    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 17:33:19.985635 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:33:19.985410    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-tls podName:2f45f9ff-9b97-4b95-8834-80308aadc3a1 nodeName:}" failed. No retries permitted until 2026-07-27 17:33:20.485391677 +0000 UTC m=+268.563824092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-tls") pod "node-exporter-kcv6b" (UID: "2f45f9ff-9b97-4b95-8834-80308aadc3a1") : secret "node-exporter-tls" not found
Jul 27 17:33:19.985635 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-textfile\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985635 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f45f9ff-9b97-4b95-8834-80308aadc3a1-sys\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985910 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f45f9ff-9b97-4b95-8834-80308aadc3a1-metrics-client-ca\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.985910 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.985810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.987762 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.987742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:19.995808 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:19.995782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddtmd\" (UniqueName: \"kubernetes.io/projected/2f45f9ff-9b97-4b95-8834-80308aadc3a1-kube-api-access-ddtmd\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:20.028395 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.028367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.128057 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.127980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.296688 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.296651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.398352 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.398262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.488696 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.488668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-tls\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:20.490978 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.490952    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2f45f9ff-9b97-4b95-8834-80308aadc3a1-node-exporter-tls\") pod \"node-exporter-kcv6b\" (UID: \"2f45f9ff-9b97-4b95-8834-80308aadc3a1\") " pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:20.756303 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.756267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kcv6b"
Jul 27 17:33:20.770345 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:20.770312    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f45f9ff_9b97_4b95_8834_80308aadc3a1.slice/crio-fafd45266dae6990c26247f85c5cc18309bb36379fbb4328b6f5374423393d35 WatchSource:0}: Error finding container fafd45266dae6990c26247f85c5cc18309bb36379fbb4328b6f5374423393d35: Status 404 returned error can't find the container with id fafd45266dae6990c26247f85c5cc18309bb36379fbb4328b6f5374423393d35
Jul 27 17:33:20.808616 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.808568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:20.884128 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:20.884090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:21.008640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:21.008562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:21.113598 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:21.113565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:21.331290 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:21.331201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kcv6b" event={"ID":"2f45f9ff-9b97-4b95-8834-80308aadc3a1","Type":"ContainerStarted","Data":"fafd45266dae6990c26247f85c5cc18309bb36379fbb4328b6f5374423393d35"}
Jul 27 17:33:22.336285 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:22.336255    2577 generic.go:356] "Generic (PLEG): container finished" podID="2f45f9ff-9b97-4b95-8834-80308aadc3a1" containerID="d2fa9d40f6b57d26aa3e4c4381d57845d85b75999c10ba8215eab156ccdc33bc" exitCode=0
Jul 27 17:33:22.336776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:22.336322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kcv6b" event={"ID":"2f45f9ff-9b97-4b95-8834-80308aadc3a1","Type":"ContainerDied","Data":"d2fa9d40f6b57d26aa3e4c4381d57845d85b75999c10ba8215eab156ccdc33bc"}
Jul 27 17:33:22.715650 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:22.715593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:22.806356 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:22.806316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:23.110746 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.110665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:23.342445 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.342402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kcv6b" event={"ID":"2f45f9ff-9b97-4b95-8834-80308aadc3a1","Type":"ContainerStarted","Data":"3bea889d8a5111fefbe8980c89a3c2134c1da9183220fc788abaecc1740cf140"}
Jul 27 17:33:23.342807 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.342473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kcv6b" event={"ID":"2f45f9ff-9b97-4b95-8834-80308aadc3a1","Type":"ContainerStarted","Data":"a96dce9561371baea7ec7d895842c8cc24c9c9138be291458b9c92dd4207f783"}
Jul 27 17:33:23.364685 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.364590    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kcv6b" podStartSLOduration=3.406980666 podStartE2EDuration="4.364573123s" totalImagesPullingTime="892.888636ms" totalInitContainerRuntime="64.703821ms" isStatefulPod=true podCreationTimestamp="2026-07-27 17:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:23.362685511 +0000 UTC m=+271.441117928" watchObservedRunningTime="2026-07-27 17:33:23.364573123 +0000 UTC m=+271.443005546"
Jul 27 17:33:23.432495 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.432468    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:23.432722 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.432695    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:23.434055 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.434030    2577 patch_prober.go:28] interesting pod/console-54595664d9-flhcq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.19:8443/health\": dial tcp 10.134.0.19:8443: connect: connection refused" start-of-body=
Jul 27 17:33:23.434132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.434073    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-54595664d9-flhcq" podUID="6964f47f-8c92-48e8-be56-03868e248815" containerName="console" probeResult="failure" output="Get \"https://10.134.0.19:8443/health\": dial tcp 10.134.0.19:8443: connect: connection refused"
Jul 27 17:33:23.584996 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:23.584963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:24.123718 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.123684    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f77db5df6-9w52q"]
Jul 27 17:33:24.126930 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.126914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.130422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.130398    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 17:33:24.130556 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.130500    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-l9nsp\""
Jul 27 17:33:24.130613 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.130563    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 17:33:24.130698 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.130685    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 17:33:24.130810 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.130792    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9em4sfhf0nenc\""
Jul 27 17:33:24.130888 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.130812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 17:33:24.135936 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.135914    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f77db5df6-9w52q"]
Jul 27 17:33:24.224198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.224198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-secret-metrics-server-tls\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.224421 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-metrics-server-audit-profiles\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.224421 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-secret-metrics-server-client-certs\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.224421 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfhwh\" (UniqueName: \"kubernetes.io/projected/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-kube-api-access-cfhwh\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.224421 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-audit-log\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.224421 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.224353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-client-ca-bundle\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.325640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-client-ca-bundle\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.325835 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.325835 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-secret-metrics-server-tls\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.325835 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-metrics-server-audit-profiles\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.325835 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-secret-metrics-server-client-certs\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.326042 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfhwh\" (UniqueName: \"kubernetes.io/projected/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-kube-api-access-cfhwh\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.326042 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.325920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-audit-log\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.326238 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.326216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-audit-log\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.326480 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.326456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.326748 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.326729    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-metrics-server-audit-profiles\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.328329 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.328298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-secret-metrics-server-tls\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.328414 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.328349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-client-ca-bundle\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.328493 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.328477    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-secret-metrics-server-client-certs\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.333423 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.333403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfhwh\" (UniqueName: \"kubernetes.io/projected/a8fc1c9e-8278-4c60-86a5-feeefdbd22a5-kube-api-access-cfhwh\") pod \"metrics-server-5f77db5df6-9w52q\" (UID: \"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5\") " pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.436740 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.436666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:24.515367 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.515336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:24.573275 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.573242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f77db5df6-9w52q"]
Jul 27 17:33:24.576721 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:24.576681    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fc1c9e_8278_4c60_86a5_feeefdbd22a5.slice/crio-54afff148188dfa9ca3eccc5896028e8d09992010e198689a1fa5eb6a6876843 WatchSource:0}: Error finding container 54afff148188dfa9ca3eccc5896028e8d09992010e198689a1fa5eb6a6876843: Status 404 returned error can't find the container with id 54afff148188dfa9ca3eccc5896028e8d09992010e198689a1fa5eb6a6876843
Jul 27 17:33:24.613676 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.613637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:24.617789 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.617747    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"]
Jul 27 17:33:24.624450 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.624415    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.626926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.626899    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 27 17:33:24.627022 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.626938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hfsvw\""
Jul 27 17:33:24.629367 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.629346    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"]
Jul 27 17:33:24.726823 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.726792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:24.729904 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.729882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kjm8\" (UniqueName: \"kubernetes.io/projected/7dd69eae-b688-4555-86b4-a22777055e8e-kube-api-access-9kjm8\") pod \"monitoring-plugin-68b7fc9574-bcrmc\" (UID: \"7dd69eae-b688-4555-86b4-a22777055e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.729960 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.729937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd69eae-b688-4555-86b4-a22777055e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bcrmc\" (UID: \"7dd69eae-b688-4555-86b4-a22777055e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.827323 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.827288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:24.830502 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.830479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd69eae-b688-4555-86b4-a22777055e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bcrmc\" (UID: \"7dd69eae-b688-4555-86b4-a22777055e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.830577 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.830565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kjm8\" (UniqueName: \"kubernetes.io/projected/7dd69eae-b688-4555-86b4-a22777055e8e-kube-api-access-9kjm8\") pod \"monitoring-plugin-68b7fc9574-bcrmc\" (UID: \"7dd69eae-b688-4555-86b4-a22777055e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.832878 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.832860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd69eae-b688-4555-86b4-a22777055e8e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bcrmc\" (UID: \"7dd69eae-b688-4555-86b4-a22777055e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.838907 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.838886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kjm8\" (UniqueName: \"kubernetes.io/projected/7dd69eae-b688-4555-86b4-a22777055e8e-kube-api-access-9kjm8\") pod \"monitoring-plugin-68b7fc9574-bcrmc\" (UID: \"7dd69eae-b688-4555-86b4-a22777055e8e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:24.934717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:24.934682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:25.060608 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.060586    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"]
Jul 27 17:33:25.062867 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:25.062841    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd69eae_b688_4555_86b4_a22777055e8e.slice/crio-fc665e70ead439200d8c8c14fcf58c7feaa89509dfc691eb1dd4f7c8d03318de WatchSource:0}: Error finding container fc665e70ead439200d8c8c14fcf58c7feaa89509dfc691eb1dd4f7c8d03318de: Status 404 returned error can't find the container with id fc665e70ead439200d8c8c14fcf58c7feaa89509dfc691eb1dd4f7c8d03318de
Jul 27 17:33:25.354313 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.354098    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f67bcb997-xpqvv"]
Jul 27 17:33:25.357628 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.357584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q" event={"ID":"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5","Type":"ContainerStarted","Data":"54afff148188dfa9ca3eccc5896028e8d09992010e198689a1fa5eb6a6876843"}
Jul 27 17:33:25.357770 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.357692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.358768 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.358731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc" event={"ID":"7dd69eae-b688-4555-86b4-a22777055e8e","Type":"ContainerStarted","Data":"fc665e70ead439200d8c8c14fcf58c7feaa89509dfc691eb1dd4f7c8d03318de"}
Jul 27 17:33:25.362455 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.362416    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f67bcb997-xpqvv"]
Jul 27 17:33:25.436663 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-oauth-serving-cert\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.436663 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-config\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.437134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-service-ca\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.437134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-oauth-config\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.437134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-trusted-ca-bundle\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.437134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-serving-cert\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.437134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.436945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mskkc\" (UniqueName: \"kubernetes.io/projected/057b5ddc-a401-4f94-b49e-efd5c103c60f-kube-api-access-mskkc\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.537625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-config\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.537812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-service-ca\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.537812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-oauth-config\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.537812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-trusted-ca-bundle\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.537812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-serving-cert\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.537812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mskkc\" (UniqueName: \"kubernetes.io/projected/057b5ddc-a401-4f94-b49e-efd5c103c60f-kube-api-access-mskkc\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.538074 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.537860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-oauth-serving-cert\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.538466 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.538412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-config\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.538466 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.538453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-service-ca\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.538792 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.538766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-oauth-serving-cert\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.538867 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.538794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-trusted-ca-bundle\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.540599 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.540573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-oauth-config\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.540976 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.540948    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-serving-cert\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.545625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.545591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mskkc\" (UniqueName: \"kubernetes.io/projected/057b5ddc-a401-4f94-b49e-efd5c103c60f-kube-api-access-mskkc\") pod \"console-f67bcb997-xpqvv\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") " pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.670888 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.670806    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:25.948895 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:25.948802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:26.119151 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.118991    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f67bcb997-xpqvv"]
Jul 27 17:33:26.121746 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:26.121713    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057b5ddc_a401_4f94_b49e_efd5c103c60f.slice/crio-16f747ef67e2ad9f29a5d6f4957be15fddd057cb3f60ae923a87ef0fc4f12251 WatchSource:0}: Error finding container 16f747ef67e2ad9f29a5d6f4957be15fddd057cb3f60ae923a87ef0fc4f12251: Status 404 returned error can't find the container with id 16f747ef67e2ad9f29a5d6f4957be15fddd057cb3f60ae923a87ef0fc4f12251
Jul 27 17:33:26.233589 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.233545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:26.291809 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.291770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:26.366123 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.366081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f67bcb997-xpqvv" event={"ID":"057b5ddc-a401-4f94-b49e-efd5c103c60f","Type":"ContainerStarted","Data":"8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741"}
Jul 27 17:33:26.366316 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.366143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f67bcb997-xpqvv" event={"ID":"057b5ddc-a401-4f94-b49e-efd5c103c60f","Type":"ContainerStarted","Data":"16f747ef67e2ad9f29a5d6f4957be15fddd057cb3f60ae923a87ef0fc4f12251"}
Jul 27 17:33:26.368018 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.367968    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q" event={"ID":"a8fc1c9e-8278-4c60-86a5-feeefdbd22a5","Type":"ContainerStarted","Data":"85bb916ccc4e66c7970f5bb376f4285970115fff7e24d397db63b209826263c0"}
Jul 27 17:33:26.384156 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.384093    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f67bcb997-xpqvv" podStartSLOduration=1.384072416 podStartE2EDuration="1.384072416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:26.382606216 +0000 UTC m=+274.461038638" watchObservedRunningTime="2026-07-27 17:33:26.384072416 +0000 UTC m=+274.462504818"
Jul 27 17:33:26.401665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:26.401610    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q" podStartSLOduration=0.954030367 podStartE2EDuration="2.401591595s" totalImagesPullingTime="1.447561228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:26.400990903 +0000 UTC m=+274.479423331" watchObservedRunningTime="2026-07-27 17:33:26.401591595 +0000 UTC m=+274.480024019"
Jul 27 17:33:27.013677 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:27.013646    2577 patch_prober.go:28] interesting pod/image-registry-d8bcdfb7f-gtf7k container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 17:33:27.013954 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:27.013711    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k" podUID="c4d74892-4cd7-451c-9b51-6d96d48f3fa7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 17:33:27.373283 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:27.373194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc" event={"ID":"7dd69eae-b688-4555-86b4-a22777055e8e","Type":"ContainerStarted","Data":"2f18f255021793f72591dceea739666064ff6efc8b35e1332a9489e97db169aa"}
Jul 27 17:33:27.389293 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:27.389247    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc" podStartSLOduration=1.486057529 podStartE2EDuration="3.389232515s" totalImagesPullingTime="1.903174986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:27.388357826 +0000 UTC m=+275.466790254" watchObservedRunningTime="2026-07-27 17:33:27.389232515 +0000 UTC m=+275.467664973"
Jul 27 17:33:28.377378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:28.377344    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:28.381796 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:28.381768    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bcrmc"
Jul 27 17:33:28.969334 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:28.969302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:29.277600 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:29.277532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d8bcdfb7f-gtf7k"
Jul 27 17:33:31.315707 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:31.315675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:31.879260 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:31.879226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:31.980006 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:31.979973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:33.436735 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:33.436702    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:33.440376 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:33.440352    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:33:33.824217 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:33.824181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:33.965661 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:33:33.965621    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t4lwh" podUID="42cb6f4f-5199-41c8-8f79-4c847ede7bcd"
Jul 27 17:33:34.399107 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:34.399078    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:33:34.566784 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:34.566741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:34.620386 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:34.620355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:34.931198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:34.931162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:35.032056 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:35.032027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:35.671866 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:35.671831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:35.672242 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:35.671898    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:35.677071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:35.677049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:36.410649 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:36.410618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:33:36.457261 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:36.457232    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54595664d9-flhcq"]
Jul 27 17:33:37.091281 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.091249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:37.191885 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.191855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:37.242210 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.242173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:33:37.244477 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.244448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42cb6f4f-5199-41c8-8f79-4c847ede7bcd-metrics-tls\") pod \"dns-default-t4lwh\" (UID: \"42cb6f4f-5199-41c8-8f79-4c847ede7bcd\") " pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:33:37.292323 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.292289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:37.403252 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.403177    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x7wp7\""
Jul 27 17:33:37.410727 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.410705    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:33:37.643017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:37.642990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t4lwh"]
Jul 27 17:33:37.645899 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:33:37.645866    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cb6f4f_5199_41c8_8f79_4c847ede7bcd.slice/crio-13ee39c9a91a22ff07da395b2e22d3003b81db7cdd2b0d05b5ec4bbc55aeb269 WatchSource:0}: Error finding container 13ee39c9a91a22ff07da395b2e22d3003b81db7cdd2b0d05b5ec4bbc55aeb269: Status 404 returned error can't find the container with id 13ee39c9a91a22ff07da395b2e22d3003b81db7cdd2b0d05b5ec4bbc55aeb269
Jul 27 17:33:38.415177 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:38.415145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t4lwh" event={"ID":"42cb6f4f-5199-41c8-8f79-4c847ede7bcd","Type":"ContainerStarted","Data":"13ee39c9a91a22ff07da395b2e22d3003b81db7cdd2b0d05b5ec4bbc55aeb269"}
Jul 27 17:33:40.109528 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:40.109505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:40.212523 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:40.212498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:40.424403 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:40.424321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t4lwh" event={"ID":"42cb6f4f-5199-41c8-8f79-4c847ede7bcd","Type":"ContainerStarted","Data":"191303990eb885b2d9d35359b05cf0649570e80457fa40fcb2d6e65a515d54c1"}
Jul 27 17:33:40.424403 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:40.424372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t4lwh" event={"ID":"42cb6f4f-5199-41c8-8f79-4c847ede7bcd","Type":"ContainerStarted","Data":"539ad42402696abdaabca522aa039833a5ce4667b50164c13947c51a79facbf0"}
Jul 27 17:33:40.424659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:40.424457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:33:40.442706 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:40.442658    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t4lwh" podStartSLOduration=251.039221055 podStartE2EDuration="4m13.442641361s" totalImagesPullingTime="2.403420306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:40.441152904 +0000 UTC m=+288.519585328" watchObservedRunningTime="2026-07-27 17:33:40.442641361 +0000 UTC m=+288.521073784"
Jul 27 17:33:44.436806 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:44.436769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:44.437181 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:44.436838    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:33:46.101418 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:46.101381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:48.392827 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:48.392793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:48.482306 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:48.482275    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:49.174934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:49.174907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:49.228824 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:49.228782    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:50.430942 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:50.430912    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t4lwh"
Jul 27 17:33:52.295692 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.295652    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:33:52.296654 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.296624    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:33:52.297145 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.297127    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:33:52.298138 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.298119    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:33:52.301454 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.301416    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:33:52.302323 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.302303    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:33:52.308554 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.308537    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 17:33:52.473126 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.473095    2577 generic.go:356] "Generic (PLEG): container finished" podID="05de6f60-7c44-4ad5-9fac-8101d7d0c3ac" containerID="ee2ed877d0f4be38a31511c6b0c7239ea4a005af0d30db890c62f219d337e663" exitCode=0
Jul 27 17:33:52.474542 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.473170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q" event={"ID":"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac","Type":"ContainerDied","Data":"ee2ed877d0f4be38a31511c6b0c7239ea4a005af0d30db890c62f219d337e663"}
Jul 27 17:33:52.474542 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:52.473524    2577 scope.go:117] "RemoveContainer" containerID="ee2ed877d0f4be38a31511c6b0c7239ea4a005af0d30db890c62f219d337e663"
Jul 27 17:33:53.479036 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:53.479004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-jht7q" event={"ID":"05de6f60-7c44-4ad5-9fac-8101d7d0c3ac","Type":"ContainerStarted","Data":"eef8f2b33a87fcbfff90394a43f20d59c08aacd36f2dcbcee89896d78df8bb81"}
Jul 27 17:33:59.553281 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:59.553249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:33:59.612940 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:33:59.612906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:01.483489 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.483397    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54595664d9-flhcq" podUID="6964f47f-8c92-48e8-be56-03868e248815" containerName="console" containerID="cri-o://b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4" gracePeriod=15
Jul 27 17:34:01.724281 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.724258    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:34:01.757290 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757217    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-oauth-serving-cert\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757290 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757257    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzz82\" (UniqueName: \"kubernetes.io/projected/6964f47f-8c92-48e8-be56-03868e248815-kube-api-access-mzz82\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757290 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-console-config\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757519 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757359    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-serving-cert\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757519 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757393    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-service-ca\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757519 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757425    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-trusted-ca-bundle\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757519 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757487    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-oauth-config\") pod \"6964f47f-8c92-48e8-be56-03868e248815\" (UID: \"6964f47f-8c92-48e8-be56-03868e248815\") "
Jul 27 17:34:01.757716 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:34:01.757761 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757716    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-oauth-serving-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:01.757869 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-service-ca" (OuterVolumeSpecName: "service-ca") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:34:01.757950 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.757920    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:34:01.758216 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.758185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-console-config" (OuterVolumeSpecName: "console-config") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:34:01.759655 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.759629    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6964f47f-8c92-48e8-be56-03868e248815-kube-api-access-mzz82" (OuterVolumeSpecName: "kube-api-access-mzz82") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "kube-api-access-mzz82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:34:01.759753 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.759704    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:34:01.759844 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.759820    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6964f47f-8c92-48e8-be56-03868e248815" (UID: "6964f47f-8c92-48e8-be56-03868e248815"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:34:01.858981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.858932    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzz82\" (UniqueName: \"kubernetes.io/projected/6964f47f-8c92-48e8-be56-03868e248815-kube-api-access-mzz82\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:01.858981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.858975    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-console-config\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:01.858981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.858985    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-serving-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:01.858981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.858993    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-service-ca\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:01.858981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.859001    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6964f47f-8c92-48e8-be56-03868e248815-trusted-ca-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:01.859273 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:01.859009    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6964f47f-8c92-48e8-be56-03868e248815-console-oauth-config\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:34:02.509898 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.509861    2577 generic.go:356] "Generic (PLEG): container finished" podID="6964f47f-8c92-48e8-be56-03868e248815" containerID="b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4" exitCode=0
Jul 27 17:34:02.510283 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.509909    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54595664d9-flhcq"
Jul 27 17:34:02.510283 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.509952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54595664d9-flhcq" event={"ID":"6964f47f-8c92-48e8-be56-03868e248815","Type":"ContainerDied","Data":"b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4"}
Jul 27 17:34:02.510283 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.510002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54595664d9-flhcq" event={"ID":"6964f47f-8c92-48e8-be56-03868e248815","Type":"ContainerDied","Data":"9eb30361691bd94751f95f3f7d72bd831f56dcbecc131eaedf7f3c1a20d37f6b"}
Jul 27 17:34:02.510283 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.510027    2577 scope.go:117] "RemoveContainer" containerID="b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4"
Jul 27 17:34:02.517974 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.517957    2577 scope.go:117] "RemoveContainer" containerID="b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4"
Jul 27 17:34:02.518218 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:34:02.518200    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4\": container with ID starting with b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4 not found: ID does not exist" containerID="b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4"
Jul 27 17:34:02.518283 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.518227    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4"} err="failed to get container status \"b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4\": rpc error: code = NotFound desc = could not find container \"b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4\": container with ID starting with b5089042d9a671ee1266cc9cdf08b903a8e485b9ac26a5ff597c48bbf0ba8ce4 not found: ID does not exist"
Jul 27 17:34:02.526290 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.526266    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54595664d9-flhcq"]
Jul 27 17:34:02.529937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:02.529919    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54595664d9-flhcq"]
Jul 27 17:34:03.360416 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:03.360386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:03.418142 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:03.418111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:04.442293 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:04.442265    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:34:04.445982 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:04.445960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f77db5df6-9w52q"
Jul 27 17:34:04.488092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:04.488059    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6964f47f-8c92-48e8-be56-03868e248815" path="/var/lib/kubelet/pods/6964f47f-8c92-48e8-be56-03868e248815/volumes"
Jul 27 17:34:13.041960 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:13.041924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:17.573287 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:17.573256    2577 generic.go:356] "Generic (PLEG): container finished" podID="ab6797ac-2bcb-45b5-94a0-6359caeff863" containerID="8eb87d8cb365227d0590a8af54d1334274b98b50b4109ee15a4669f39591056d" exitCode=0
Jul 27 17:34:17.573666 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:17.573328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jvqln" event={"ID":"ab6797ac-2bcb-45b5-94a0-6359caeff863","Type":"ContainerDied","Data":"8eb87d8cb365227d0590a8af54d1334274b98b50b4109ee15a4669f39591056d"}
Jul 27 17:34:17.573712 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:17.573680    2577 scope.go:117] "RemoveContainer" containerID="8eb87d8cb365227d0590a8af54d1334274b98b50b4109ee15a4669f39591056d"
Jul 27 17:34:18.578718 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:18.578681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jvqln" event={"ID":"ab6797ac-2bcb-45b5-94a0-6359caeff863","Type":"ContainerStarted","Data":"47e809b46e073da1b0b3a19c26bba5a17d12500e6c9b1bc77512b19aa1e1efa5"}
Jul 27 17:34:39.012644 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:39.012601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:39.514446 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:39.514401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:44.093422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.093336    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-d5dd789bc-gkd27"]
Jul 27 17:34:44.093910 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.093854    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6964f47f-8c92-48e8-be56-03868e248815" containerName="console"
Jul 27 17:34:44.093910 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.093872    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6964f47f-8c92-48e8-be56-03868e248815" containerName="console"
Jul 27 17:34:44.094017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.093977    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6964f47f-8c92-48e8-be56-03868e248815" containerName="console"
Jul 27 17:34:44.097231 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.097207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.099633 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.099611    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 27 17:34:44.099633 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.099623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 27 17:34:44.099831 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.099667    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 27 17:34:44.099958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.099937    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 27 17:34:44.100079 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.100047    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 27 17:34:44.100079 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.100071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v4sp7\""
Jul 27 17:34:44.105114 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.105092    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 27 17:34:44.107391 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.107371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d5dd789bc-gkd27"]
Jul 27 17:34:44.206999 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.206965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-federate-client-tls\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-metrics-client-ca\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-telemeter-client-tls\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-serving-certs-ca-bundle\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vst6g\" (UniqueName: \"kubernetes.io/projected/a6890efe-917b-47ef-af71-c60c6e895c04-kube-api-access-vst6g\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.207405 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.207189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-secret-telemeter-client\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.307877 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.307842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-telemeter-client-tls\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308052 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.307883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-serving-certs-ca-bundle\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308052 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.307913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308052 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.307944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308052 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.307978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vst6g\" (UniqueName: \"kubernetes.io/projected/a6890efe-917b-47ef-af71-c60c6e895c04-kube-api-access-vst6g\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308052 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.308001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-secret-telemeter-client\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308316 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.308066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-federate-client-tls\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308316 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.308108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-metrics-client-ca\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308803 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.308779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-serving-certs-ca-bundle\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.308973 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.308944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-metrics-client-ca\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.309102 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.309035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6890efe-917b-47ef-af71-c60c6e895c04-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.310655 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.310633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-federate-client-tls\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.310736 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.310638    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-secret-telemeter-client\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.310820 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.310804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.310987 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.310967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a6890efe-917b-47ef-af71-c60c6e895c04-telemeter-client-tls\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.315866 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.315843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vst6g\" (UniqueName: \"kubernetes.io/projected/a6890efe-917b-47ef-af71-c60c6e895c04-kube-api-access-vst6g\") pod \"telemeter-client-d5dd789bc-gkd27\" (UID: \"a6890efe-917b-47ef-af71-c60c6e895c04\") " pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.407801 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.407710    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27"
Jul 27 17:34:44.545667 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.545642    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d5dd789bc-gkd27"]
Jul 27 17:34:44.547774 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:34:44.547732    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6890efe_917b_47ef_af71_c60c6e895c04.slice/crio-244781b92037d07c40cd63b627dabedd122d1819fe377950dcc0ed03f05065ce WatchSource:0}: Error finding container 244781b92037d07c40cd63b627dabedd122d1819fe377950dcc0ed03f05065ce: Status 404 returned error can't find the container with id 244781b92037d07c40cd63b627dabedd122d1819fe377950dcc0ed03f05065ce
Jul 27 17:34:44.549910 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.549890    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:34:44.674904 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:44.674827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27" event={"ID":"a6890efe-917b-47ef-af71-c60c6e895c04","Type":"ContainerStarted","Data":"244781b92037d07c40cd63b627dabedd122d1819fe377950dcc0ed03f05065ce"}
Jul 27 17:34:45.507532 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:45.507488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:45.607735 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:45.607701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:46.683759 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:46.683726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27" event={"ID":"a6890efe-917b-47ef-af71-c60c6e895c04","Type":"ContainerStarted","Data":"89eaee8ac057665b0767aa889b51d350de5276da55f1cdc8a59fa70d8a05b159"}
Jul 27 17:34:47.690868 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:47.690838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27" event={"ID":"a6890efe-917b-47ef-af71-c60c6e895c04","Type":"ContainerStarted","Data":"2fd991be269d3407e54899760aba333602a2c287ddc2b9b055080d260cd7827f"}
Jul 27 17:34:48.696643 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:48.696608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27" event={"ID":"a6890efe-917b-47ef-af71-c60c6e895c04","Type":"ContainerStarted","Data":"b6d340ef2e2e7dd21cff6fed2e422c25111d2e61917710417c5b38922ccad475"}
Jul 27 17:34:48.719789 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:48.719737    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-d5dd789bc-gkd27" podStartSLOduration=1.7292666589999999 podStartE2EDuration="4.719720909s" totalImagesPullingTime="2.99045425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:34:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:34:48.718073762 +0000 UTC m=+356.796506186" watchObservedRunningTime="2026-07-27 17:34:48.719720909 +0000 UTC m=+356.798153332"
Jul 27 17:34:49.476873 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.476841    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bf76f894d-jt5rt"]
Jul 27 17:34:49.480574 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.480548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.492600 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.492574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bf76f894d-jt5rt"]
Jul 27 17:34:49.536200 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.536168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:49.590776 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.590746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:49.657924 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.657899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-oauth-config\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.658071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.657951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-serving-cert\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.658071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.657969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-service-ca\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.658071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.657984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db6rj\" (UniqueName: \"kubernetes.io/projected/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-kube-api-access-db6rj\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.658071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.658003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-trusted-ca-bundle\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.658198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.658068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-config\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.658198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.658101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-oauth-serving-cert\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.758995 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.758905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-config\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.758995 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.758959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-oauth-serving-cert\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.758995 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.758988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-oauth-config\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759601 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-serving-cert\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759601 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-service-ca\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759601 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db6rj\" (UniqueName: \"kubernetes.io/projected/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-kube-api-access-db6rj\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759601 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-trusted-ca-bundle\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759823 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-config\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759880 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-oauth-serving-cert\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.759880 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.759838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-service-ca\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.760040 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.760023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-trusted-ca-bundle\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.761535 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.761509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-serving-cert\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.761535 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.761524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-oauth-config\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.767365 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.767345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db6rj\" (UniqueName: \"kubernetes.io/projected/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-kube-api-access-db6rj\") pod \"console-5bf76f894d-jt5rt\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") " pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.790122 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.790099    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:49.918240 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:49.918206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bf76f894d-jt5rt"]
Jul 27 17:34:49.920910 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:34:49.920885    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47aced2_cd90_4d86_9ecc_4aa38d0aa53e.slice/crio-248265a26f898bd317d4dee108468c0c68792dd6b5ae0062177cde7ba46e8e28 WatchSource:0}: Error finding container 248265a26f898bd317d4dee108468c0c68792dd6b5ae0062177cde7ba46e8e28: Status 404 returned error can't find the container with id 248265a26f898bd317d4dee108468c0c68792dd6b5ae0062177cde7ba46e8e28
Jul 27 17:34:50.705124 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:50.705083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf76f894d-jt5rt" event={"ID":"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e","Type":"ContainerStarted","Data":"863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2"}
Jul 27 17:34:50.705302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:50.705136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf76f894d-jt5rt" event={"ID":"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e","Type":"ContainerStarted","Data":"248265a26f898bd317d4dee108468c0c68792dd6b5ae0062177cde7ba46e8e28"}
Jul 27 17:34:50.723499 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:50.723452    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bf76f894d-jt5rt" podStartSLOduration=1.7234123609999998 podStartE2EDuration="1.723412361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:34:50.722261664 +0000 UTC m=+358.800694144" watchObservedRunningTime="2026-07-27 17:34:50.723412361 +0000 UTC m=+358.801844784"
Jul 27 17:34:54.188640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:54.188601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:54.245314 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:54.245280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:54.745884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:54.745852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:54.800787 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:54.800754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:34:59.757710 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.757674    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bf76f894d-jt5rt"]
Jul 27 17:34:59.785336 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.785302    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68d5c4bcf9-b6sdd"]
Jul 27 17:34:59.791824 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.791801    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:34:59.791952 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.791885    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.798078 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.798047    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68d5c4bcf9-b6sdd"]
Jul 27 17:34:59.834622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-oauth-serving-cert\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.834788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-service-ca\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.834788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29ppg\" (UniqueName: \"kubernetes.io/projected/f24341b9-4f61-4379-b851-243a01798834-kube-api-access-29ppg\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.834788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24341b9-4f61-4379-b851-243a01798834-console-serving-cert\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.834788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-console-config\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.834788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-trusted-ca-bundle\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.834941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.834821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24341b9-4f61-4379-b851-243a01798834-console-oauth-config\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935234 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-oauth-serving-cert\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935234 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-service-ca\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935527 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29ppg\" (UniqueName: \"kubernetes.io/projected/f24341b9-4f61-4379-b851-243a01798834-kube-api-access-29ppg\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935527 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24341b9-4f61-4379-b851-243a01798834-console-serving-cert\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935527 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-console-config\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935527 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-trusted-ca-bundle\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.935527 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.935502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24341b9-4f61-4379-b851-243a01798834-console-oauth-config\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.936125 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.936091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-oauth-serving-cert\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.936239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.936097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-service-ca\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.936239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.936151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-console-config\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.936411 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.936389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24341b9-4f61-4379-b851-243a01798834-trusted-ca-bundle\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.937981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.937961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24341b9-4f61-4379-b851-243a01798834-console-serving-cert\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.937981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.937978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24341b9-4f61-4379-b851-243a01798834-console-oauth-config\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:34:59.943230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:34:59.943209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29ppg\" (UniqueName: \"kubernetes.io/projected/f24341b9-4f61-4379-b851-243a01798834-kube-api-access-29ppg\") pod \"console-68d5c4bcf9-b6sdd\" (UID: \"f24341b9-4f61-4379-b851-243a01798834\") " pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:35:00.103567 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:00.103488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:35:00.233594 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:00.233567    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68d5c4bcf9-b6sdd"]
Jul 27 17:35:00.236253 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:35:00.236225    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24341b9_4f61_4379_b851_243a01798834.slice/crio-c901d05198c4e8f7b0264e688ba2db9219b1701d2efeb2d3a8df0115b0dca871 WatchSource:0}: Error finding container c901d05198c4e8f7b0264e688ba2db9219b1701d2efeb2d3a8df0115b0dca871: Status 404 returned error can't find the container with id c901d05198c4e8f7b0264e688ba2db9219b1701d2efeb2d3a8df0115b0dca871
Jul 27 17:35:00.743876 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:00.743837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d5c4bcf9-b6sdd" event={"ID":"f24341b9-4f61-4379-b851-243a01798834","Type":"ContainerStarted","Data":"fe5f4f8509d261ef59af33e5f8721c233931895e903e1e34a4f6b139a5b37e8c"}
Jul 27 17:35:00.744055 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:00.743891    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d5c4bcf9-b6sdd" event={"ID":"f24341b9-4f61-4379-b851-243a01798834","Type":"ContainerStarted","Data":"c901d05198c4e8f7b0264e688ba2db9219b1701d2efeb2d3a8df0115b0dca871"}
Jul 27 17:35:00.765445 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:00.765392    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68d5c4bcf9-b6sdd" podStartSLOduration=1.765375871 podStartE2EDuration="1.765375871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:35:00.760297374 +0000 UTC m=+368.838729797" watchObservedRunningTime="2026-07-27 17:35:00.765375871 +0000 UTC m=+368.843808296"
Jul 27 17:35:07.139406 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:07.139366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:07.202867 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:07.202833    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:10.104256 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.104216    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:35:10.104830 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.104270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:35:10.109115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.109093    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:35:10.769268 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.769236    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:10.791205 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.791181    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68d5c4bcf9-b6sdd"
Jul 27 17:35:10.840396 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.840362    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f67bcb997-xpqvv"]
Jul 27 17:35:10.947616 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:10.947581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:11.051802 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:11.051728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:11.151869 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:11.151841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:18.370307 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:18.370272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:18.477669 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:18.477629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:35:24.777794 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:24.777749    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bf76f894d-jt5rt" podUID="d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" containerName="console" containerID="cri-o://863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2" gracePeriod=15
Jul 27 17:35:25.033642 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.033578    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:35:25.146055 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146008    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-oauth-serving-cert\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146253 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146074    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-service-ca\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146253 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146106    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-serving-cert\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146253 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db6rj\" (UniqueName: \"kubernetes.io/projected/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-kube-api-access-db6rj\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146253 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146197    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-oauth-config\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146253 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146241    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-config\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146534 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146297    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-trusted-ca-bundle\") pod \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\" (UID: \"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e\") "
Jul 27 17:35:25.146534 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146464    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:25.146623 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146563    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-oauth-serving-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.146680 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146556    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-service-ca" (OuterVolumeSpecName: "service-ca") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:25.146831 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146801    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:25.146908 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.146845    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-config" (OuterVolumeSpecName: "console-config") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:25.148280 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.148253    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:35:25.148370 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.148276    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:35:25.148370 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.148293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-kube-api-access-db6rj" (OuterVolumeSpecName: "kube-api-access-db6rj") pod "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" (UID: "d47aced2-cd90-4d86-9ecc-4aa38d0aa53e"). InnerVolumeSpecName "kube-api-access-db6rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:35:25.247801 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.247768    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-service-ca\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.247801 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.247795    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-serving-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.247801 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.247805    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db6rj\" (UniqueName: \"kubernetes.io/projected/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-kube-api-access-db6rj\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.248015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.247815    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-oauth-config\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.248015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.247823    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-console-config\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.248015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.247831    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e-trusted-ca-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:25.845840 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.845803    2577 generic.go:356] "Generic (PLEG): container finished" podID="d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" containerID="863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2" exitCode=0
Jul 27 17:35:25.846232 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.845849    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bf76f894d-jt5rt"
Jul 27 17:35:25.846232 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.845874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf76f894d-jt5rt" event={"ID":"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e","Type":"ContainerDied","Data":"863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2"}
Jul 27 17:35:25.846232 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.845931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bf76f894d-jt5rt" event={"ID":"d47aced2-cd90-4d86-9ecc-4aa38d0aa53e","Type":"ContainerDied","Data":"248265a26f898bd317d4dee108468c0c68792dd6b5ae0062177cde7ba46e8e28"}
Jul 27 17:35:25.846232 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.845952    2577 scope.go:117] "RemoveContainer" containerID="863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2"
Jul 27 17:35:25.855409 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.855396    2577 scope.go:117] "RemoveContainer" containerID="863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2"
Jul 27 17:35:25.855666 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:35:25.855648    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2\": container with ID starting with 863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2 not found: ID does not exist" containerID="863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2"
Jul 27 17:35:25.855716 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.855674    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2"} err="failed to get container status \"863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2\": rpc error: code = NotFound desc = could not find container \"863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2\": container with ID starting with 863b8ad11eba31fe408f4e99a6ee8bf4eaf60d24a28d1a8793972e3a00adc9c2 not found: ID does not exist"
Jul 27 17:35:25.866195 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.866170    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bf76f894d-jt5rt"]
Jul 27 17:35:25.870208 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:25.870188    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bf76f894d-jt5rt"]
Jul 27 17:35:26.483369 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:26.483332    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" path="/var/lib/kubelet/pods/d47aced2-cd90-4d86-9ecc-4aa38d0aa53e/volumes"
Jul 27 17:35:35.861254 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:35.861201    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f67bcb997-xpqvv" podUID="057b5ddc-a401-4f94-b49e-efd5c103c60f" containerName="console" containerID="cri-o://8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741" gracePeriod=15
Jul 27 17:35:36.121788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.121732    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:35:36.236848 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.236810    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-service-ca\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237006 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.236854    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-config\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237006 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.236914    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-trusted-ca-bundle\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237006 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.236970    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mskkc\" (UniqueName: \"kubernetes.io/projected/057b5ddc-a401-4f94-b49e-efd5c103c60f-kube-api-access-mskkc\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237006 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237002    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-oauth-serving-cert\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237228 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237038    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-serving-cert\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237228 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237080    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-oauth-config\") pod \"057b5ddc-a401-4f94-b49e-efd5c103c60f\" (UID: \"057b5ddc-a401-4f94-b49e-efd5c103c60f\") "
Jul 27 17:35:36.237325 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-config" (OuterVolumeSpecName: "console-config") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:36.237325 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237297    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-service-ca" (OuterVolumeSpecName: "service-ca") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:36.237422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237355    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-service-ca\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.237422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237374    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-config\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.237422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:36.237422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.237402    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 17:35:36.239216 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.239187    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/057b5ddc-a401-4f94-b49e-efd5c103c60f-kube-api-access-mskkc" (OuterVolumeSpecName: "kube-api-access-mskkc") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "kube-api-access-mskkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:35:36.239216 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.239211    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:35:36.239346 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.239196    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "057b5ddc-a401-4f94-b49e-efd5c103c60f" (UID: "057b5ddc-a401-4f94-b49e-efd5c103c60f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:35:36.338472 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.338409    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-trusted-ca-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.338472 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.338464    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mskkc\" (UniqueName: \"kubernetes.io/projected/057b5ddc-a401-4f94-b49e-efd5c103c60f-kube-api-access-mskkc\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.338472 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.338485    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/057b5ddc-a401-4f94-b49e-efd5c103c60f-oauth-serving-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.338718 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.338494    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-serving-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.338718 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.338503    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/057b5ddc-a401-4f94-b49e-efd5c103c60f-console-oauth-config\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:35:36.889920 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.889831    2577 generic.go:356] "Generic (PLEG): container finished" podID="057b5ddc-a401-4f94-b49e-efd5c103c60f" containerID="8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741" exitCode=0
Jul 27 17:35:36.889920 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.889881    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f67bcb997-xpqvv"
Jul 27 17:35:36.890332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.889917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f67bcb997-xpqvv" event={"ID":"057b5ddc-a401-4f94-b49e-efd5c103c60f","Type":"ContainerDied","Data":"8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741"}
Jul 27 17:35:36.890332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.889970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f67bcb997-xpqvv" event={"ID":"057b5ddc-a401-4f94-b49e-efd5c103c60f","Type":"ContainerDied","Data":"16f747ef67e2ad9f29a5d6f4957be15fddd057cb3f60ae923a87ef0fc4f12251"}
Jul 27 17:35:36.890332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.889990    2577 scope.go:117] "RemoveContainer" containerID="8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741"
Jul 27 17:35:36.903864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.903843    2577 scope.go:117] "RemoveContainer" containerID="8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741"
Jul 27 17:35:36.904143 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:35:36.904123    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741\": container with ID starting with 8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741 not found: ID does not exist" containerID="8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741"
Jul 27 17:35:36.904189 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.904152    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741"} err="failed to get container status \"8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741\": rpc error: code = NotFound desc = could not find container \"8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741\": container with ID starting with 8f0f3779bb9cce46fa5447215bde8850c70db6ead3a5b07acb6ae97f87510741 not found: ID does not exist"
Jul 27 17:35:36.909203 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.909178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f67bcb997-xpqvv"]
Jul 27 17:35:36.912605 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:36.912580    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f67bcb997-xpqvv"]
Jul 27 17:35:38.482716 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:35:38.482683    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057b5ddc-a401-4f94-b49e-efd5c103c60f" path="/var/lib/kubelet/pods/057b5ddc-a401-4f94-b49e-efd5c103c60f/volumes"
Jul 27 17:36:18.877376 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:36:18.877290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:42.130299 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:36:42.130266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:42.231538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:36:42.231504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:42.451606 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:36:42.451534    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:42.554057 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:36:42.554019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:36:42.655079 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:36:42.655047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:04.617110 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:04.617076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:04.694005 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:04.693978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:05.671646 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.671608    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"]
Jul 27 17:37:05.672113 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.672033    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057b5ddc-a401-4f94-b49e-efd5c103c60f" containerName="console"
Jul 27 17:37:05.672113 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.672053    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="057b5ddc-a401-4f94-b49e-efd5c103c60f" containerName="console"
Jul 27 17:37:05.672113 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.672072    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" containerName="console"
Jul 27 17:37:05.672113 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.672080    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" containerName="console"
Jul 27 17:37:05.672353 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.672177    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="057b5ddc-a401-4f94-b49e-efd5c103c60f" containerName="console"
Jul 27 17:37:05.672353 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.672191    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47aced2-cd90-4d86-9ecc-4aa38d0aa53e" containerName="console"
Jul 27 17:37:05.675195 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.675173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.678188 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.678170    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-8z5kp\""
Jul 27 17:37:05.678577 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.678548    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 27 17:37:05.678736 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.678722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 27 17:37:05.678812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.678753    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 27 17:37:05.688654 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.688635    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"]
Jul 27 17:37:05.770806 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.770771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl22l\" (UniqueName: \"kubernetes.io/projected/d1ca9fa5-fb35-4c3c-945b-ff10276367de-kube-api-access-xl22l\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6\" (UID: \"d1ca9fa5-fb35-4c3c-945b-ff10276367de\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.770981 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.770843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d1ca9fa5-fb35-4c3c-945b-ff10276367de-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6\" (UID: \"d1ca9fa5-fb35-4c3c-945b-ff10276367de\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.871836 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.871803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d1ca9fa5-fb35-4c3c-945b-ff10276367de-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6\" (UID: \"d1ca9fa5-fb35-4c3c-945b-ff10276367de\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.871952 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.871885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl22l\" (UniqueName: \"kubernetes.io/projected/d1ca9fa5-fb35-4c3c-945b-ff10276367de-kube-api-access-xl22l\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6\" (UID: \"d1ca9fa5-fb35-4c3c-945b-ff10276367de\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.874181 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.874161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d1ca9fa5-fb35-4c3c-945b-ff10276367de-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6\" (UID: \"d1ca9fa5-fb35-4c3c-945b-ff10276367de\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.880954 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.880933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl22l\" (UniqueName: \"kubernetes.io/projected/d1ca9fa5-fb35-4c3c-945b-ff10276367de-kube-api-access-xl22l\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6\" (UID: \"d1ca9fa5-fb35-4c3c-945b-ff10276367de\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:05.984995 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:05.984962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:06.117076 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:06.117053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"]
Jul 27 17:37:06.119375 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:37:06.119347    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1ca9fa5_fb35_4c3c_945b_ff10276367de.slice/crio-0c5921b15bf26a3582020a63a4f29efbdb9794b30f417557ef3034d877f94ee4 WatchSource:0}: Error finding container 0c5921b15bf26a3582020a63a4f29efbdb9794b30f417557ef3034d877f94ee4: Status 404 returned error can't find the container with id 0c5921b15bf26a3582020a63a4f29efbdb9794b30f417557ef3034d877f94ee4
Jul 27 17:37:06.238178 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:06.238094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6" event={"ID":"d1ca9fa5-fb35-4c3c-945b-ff10276367de","Type":"ContainerStarted","Data":"0c5921b15bf26a3582020a63a4f29efbdb9794b30f417557ef3034d877f94ee4"}
Jul 27 17:37:10.260380 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.260345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6" event={"ID":"d1ca9fa5-fb35-4c3c-945b-ff10276367de","Type":"ContainerStarted","Data":"c7b83fd730879f71347e058396d9939ba4b1150ed1b22c364ecdc42dacc376bf"}
Jul 27 17:37:10.260789 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.260475    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:10.283384 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.283320    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6" podStartSLOduration=1.577394035 podStartE2EDuration="5.283301155s" totalImagesPullingTime="3.70590712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:37:10.281769202 +0000 UTC m=+498.360201625" watchObservedRunningTime="2026-07-27 17:37:10.283301155 +0000 UTC m=+498.361733579"
Jul 27 17:37:10.327794 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.327759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:10.594649 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.594564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:10.598315 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.598287    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-tnpgh"]
Jul 27 17:37:10.601657 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.601636    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.604612 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.604589    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-5cqvh\""
Jul 27 17:37:10.605122 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.605105    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 27 17:37:10.605223 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.605206    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 27 17:37:10.611423 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.611403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-tnpgh"]
Jul 27 17:37:10.716989 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.716949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cd65373c-42be-4a07-bafa-8c2f76100ddf-cabundle0\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.717152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.717070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.717152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.717107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjcx\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-kube-api-access-nhjcx\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.800568 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.800533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:10.801123 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.801098    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"]
Jul 27 17:37:10.804250 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.804231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:10.806640 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.806621    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 27 17:37:10.813595 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.813553    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"]
Jul 27 17:37:10.817869 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.817846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.817962 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.817888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjcx\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-kube-api-access-nhjcx\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.817962 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.817950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cd65373c-42be-4a07-bafa-8c2f76100ddf-cabundle0\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.818041 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:10.817982    2577 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 27 17:37:10.818041 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:10.818002    2577 secret.go:281] references non-existent secret key: ca.crt
Jul 27 17:37:10.818041 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:10.818013    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 27 17:37:10.818041 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:10.818028    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-tnpgh: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 27 17:37:10.818190 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:10.818087    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates podName:cd65373c-42be-4a07-bafa-8c2f76100ddf nodeName:}" failed. No retries permitted until 2026-07-27 17:37:11.318069316 +0000 UTC m=+499.396501732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates") pod "keda-operator-77f648d694-tnpgh" (UID: "cd65373c-42be-4a07-bafa-8c2f76100ddf") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 27 17:37:10.818739 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.818719    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cd65373c-42be-4a07-bafa-8c2f76100ddf-cabundle0\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.827227 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.827201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjcx\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-kube-api-access-nhjcx\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:10.919115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.919025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m64vn\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-kube-api-access-m64vn\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:10.919115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.919069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7e8c6903-ec1b-485c-8b4a-45446eac340b-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:10.919115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.919100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-certificates\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:10.944197 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:10.944165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:11.019748 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.019717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m64vn\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-kube-api-access-m64vn\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.019922 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.019759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7e8c6903-ec1b-485c-8b4a-45446eac340b-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.019922 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.019788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-certificates\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.019922 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.019896    2577 secret.go:281] references non-existent secret key: tls.crt
Jul 27 17:37:11.019922 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.019909    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 27 17:37:11.020121 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.019927    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg: references non-existent secret key: tls.crt
Jul 27 17:37:11.020121 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.020000    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-certificates podName:7e8c6903-ec1b-485c-8b4a-45446eac340b nodeName:}" failed. No retries permitted until 2026-07-27 17:37:11.519974303 +0000 UTC m=+499.598406709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-certificates") pod "keda-metrics-apiserver-55557df8c-tgrjg" (UID: "7e8c6903-ec1b-485c-8b4a-45446eac340b") : references non-existent secret key: tls.crt
Jul 27 17:37:11.020234 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.020146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7e8c6903-ec1b-485c-8b4a-45446eac340b-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.029019 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.028996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m64vn\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-kube-api-access-m64vn\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.181583 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.181507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:11.281895 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.281847    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:11.322874 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.322836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:11.323062 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.322936    2577 secret.go:281] references non-existent secret key: ca.crt
Jul 27 17:37:11.323062 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.322956    2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 27 17:37:11.323062 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.322965    2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-tnpgh: references non-existent secret key: ca.crt
Jul 27 17:37:11.323062 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:37:11.323034    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates podName:cd65373c-42be-4a07-bafa-8c2f76100ddf nodeName:}" failed. No retries permitted until 2026-07-27 17:37:12.323018586 +0000 UTC m=+500.401450986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates") pod "keda-operator-77f648d694-tnpgh" (UID: "cd65373c-42be-4a07-bafa-8c2f76100ddf") : references non-existent secret key: ca.crt
Jul 27 17:37:11.382265 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.382230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:11.389319 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.389292    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-7qbjb"]
Jul 27 17:37:11.392789 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.392774    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.395537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.395518    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 27 17:37:11.402248 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.402225    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-7qbjb"]
Jul 27 17:37:11.524332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.524274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8d807fd4-8021-4fae-8bdd-31668e278945-certificates\") pod \"keda-admission-56f9798bb4-7qbjb\" (UID: \"8d807fd4-8021-4fae-8bdd-31668e278945\") " pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.524552 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.524362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-certificates\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.524552 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.524422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxwbb\" (UniqueName: \"kubernetes.io/projected/8d807fd4-8021-4fae-8bdd-31668e278945-kube-api-access-cxwbb\") pod \"keda-admission-56f9798bb4-7qbjb\" (UID: \"8d807fd4-8021-4fae-8bdd-31668e278945\") " pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.526822 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.526795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e8c6903-ec1b-485c-8b4a-45446eac340b-certificates\") pod \"keda-metrics-apiserver-55557df8c-tgrjg\" (UID: \"7e8c6903-ec1b-485c-8b4a-45446eac340b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.552606 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.552573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:11.625641 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.625609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8d807fd4-8021-4fae-8bdd-31668e278945-certificates\") pod \"keda-admission-56f9798bb4-7qbjb\" (UID: \"8d807fd4-8021-4fae-8bdd-31668e278945\") " pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.625805 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.625679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxwbb\" (UniqueName: \"kubernetes.io/projected/8d807fd4-8021-4fae-8bdd-31668e278945-kube-api-access-cxwbb\") pod \"keda-admission-56f9798bb4-7qbjb\" (UID: \"8d807fd4-8021-4fae-8bdd-31668e278945\") " pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.628189 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.628158    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8d807fd4-8021-4fae-8bdd-31668e278945-certificates\") pod \"keda-admission-56f9798bb4-7qbjb\" (UID: \"8d807fd4-8021-4fae-8bdd-31668e278945\") " pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.640763 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.640734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxwbb\" (UniqueName: \"kubernetes.io/projected/8d807fd4-8021-4fae-8bdd-31668e278945-kube-api-access-cxwbb\") pod \"keda-admission-56f9798bb4-7qbjb\" (UID: \"8d807fd4-8021-4fae-8bdd-31668e278945\") " pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.704351 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.704311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:11.715098 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.714937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:11.755095 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.755058    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:11.853131 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.853106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-7qbjb"]
Jul 27 17:37:11.855758 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:37:11.855724    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d807fd4_8021_4fae_8bdd_31668e278945.slice/crio-be043980ca802cfce7b73a6b9762aef65e5132da28d2a0960d27bf0dbf4c2f1d WatchSource:0}: Error finding container be043980ca802cfce7b73a6b9762aef65e5132da28d2a0960d27bf0dbf4c2f1d: Status 404 returned error can't find the container with id be043980ca802cfce7b73a6b9762aef65e5132da28d2a0960d27bf0dbf4c2f1d
Jul 27 17:37:11.877922 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.877896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"]
Jul 27 17:37:11.880090 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:37:11.880068    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8c6903_ec1b_485c_8b4a_45446eac340b.slice/crio-d11f374e58279cb11e17ec14e12e05ae5ec9b374d63f99db8f1fac3daa0df21a WatchSource:0}: Error finding container d11f374e58279cb11e17ec14e12e05ae5ec9b374d63f99db8f1fac3daa0df21a: Status 404 returned error can't find the container with id d11f374e58279cb11e17ec14e12e05ae5ec9b374d63f99db8f1fac3daa0df21a
Jul 27 17:37:11.946154 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:11.946113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:12.083650 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.083565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:12.184067 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.184022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:37:12.269807 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.269776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg" event={"ID":"7e8c6903-ec1b-485c-8b4a-45446eac340b","Type":"ContainerStarted","Data":"d11f374e58279cb11e17ec14e12e05ae5ec9b374d63f99db8f1fac3daa0df21a"}
Jul 27 17:37:12.270882 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.270861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb" event={"ID":"8d807fd4-8021-4fae-8bdd-31668e278945","Type":"ContainerStarted","Data":"be043980ca802cfce7b73a6b9762aef65e5132da28d2a0960d27bf0dbf4c2f1d"}
Jul 27 17:37:12.332790 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.332752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:12.335487 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.335407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd65373c-42be-4a07-bafa-8c2f76100ddf-certificates\") pod \"keda-operator-77f648d694-tnpgh\" (UID: \"cd65373c-42be-4a07-bafa-8c2f76100ddf\") " pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:12.412380 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.412352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:12.553927 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:12.553905    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-tnpgh"]
Jul 27 17:37:12.556445 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:37:12.556401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd65373c_42be_4a07_bafa_8c2f76100ddf.slice/crio-a354a1893a3210ce2c74c8af2d4668cb5a140883cce96f793294afecbb6670bd WatchSource:0}: Error finding container a354a1893a3210ce2c74c8af2d4668cb5a140883cce96f793294afecbb6670bd: Status 404 returned error can't find the container with id a354a1893a3210ce2c74c8af2d4668cb5a140883cce96f793294afecbb6670bd
Jul 27 17:37:13.276941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:13.276903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-tnpgh" event={"ID":"cd65373c-42be-4a07-bafa-8c2f76100ddf","Type":"ContainerStarted","Data":"a354a1893a3210ce2c74c8af2d4668cb5a140883cce96f793294afecbb6670bd"}
Jul 27 17:37:15.288193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:15.288161    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg" event={"ID":"7e8c6903-ec1b-485c-8b4a-45446eac340b","Type":"ContainerStarted","Data":"19102d2cf59929c260b98e018f1ee94bede7ec85a2edc59eee6123ecf915088a"}
Jul 27 17:37:15.288661 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:15.288320    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:15.289704 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:15.289678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb" event={"ID":"8d807fd4-8021-4fae-8bdd-31668e278945","Type":"ContainerStarted","Data":"e75a808db5ca85caebefa617f74648b1de98f034c2ee7816efc78280a77a3ad7"}
Jul 27 17:37:15.289824 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:15.289795    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:15.306301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:15.306253    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg" podStartSLOduration=2.453193867 podStartE2EDuration="5.306239756s" totalImagesPullingTime="2.853045889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:37:15.306013092 +0000 UTC m=+503.384445516" watchObservedRunningTime="2026-07-27 17:37:15.306239756 +0000 UTC m=+503.384672176"
Jul 27 17:37:15.322839 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:15.322793    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb" podStartSLOduration=1.445812948 podStartE2EDuration="4.322778955s" totalImagesPullingTime="2.876966007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:37:15.321500417 +0000 UTC m=+503.399932841" watchObservedRunningTime="2026-07-27 17:37:15.322778955 +0000 UTC m=+503.401211378"
Jul 27 17:37:18.304714 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:18.304681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-tnpgh" event={"ID":"cd65373c-42be-4a07-bafa-8c2f76100ddf","Type":"ContainerStarted","Data":"83155481ce4cb857a7bb665000f60aa24b413477c30ff24a3dc5cb13f6a7d257"}
Jul 27 17:37:18.305145 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:18.304763    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:37:18.322962 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:18.322917    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-tnpgh" podStartSLOduration=3.209501652 podStartE2EDuration="8.322902793s" totalImagesPullingTime="5.113401141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:37:18.321675419 +0000 UTC m=+506.400107842" watchObservedRunningTime="2026-07-27 17:37:18.322902793 +0000 UTC m=+506.401335216"
Jul 27 17:37:26.299774 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:26.299747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-tgrjg"
Jul 27 17:37:31.267452 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:31.267398    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-mgpd6"
Jul 27 17:37:36.297687 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:36.297654    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-7qbjb"
Jul 27 17:37:39.311117 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:37:39.311088    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-tnpgh"
Jul 27 17:38:26.205111 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.205077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.290949 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.290912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.291941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.291917    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-7584bddd84-vvhxz"]
Jul 27 17:38:26.294699 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.294677    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.297483 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.297464    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 27 17:38:26.297727 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.297713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 27 17:38:26.298568 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.298549    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 27 17:38:26.298780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.298568    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-zgd9l\""
Jul 27 17:38:26.304604 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.304584    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-7584bddd84-vvhxz"]
Jul 27 17:38:26.367056 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.367023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t796c\" (UniqueName: \"kubernetes.io/projected/61b5d9bd-868f-48f1-9a82-0729a06be2dd-kube-api-access-t796c\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.367198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.367086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.372144 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.372097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.467734 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.467710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.467929 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:38:26.467868    2577 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 27 17:38:26.467929 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:38:26.467927    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert podName:61b5d9bd-868f-48f1-9a82-0729a06be2dd nodeName:}" failed. No retries permitted until 2026-07-27 17:38:26.967911948 +0000 UTC m=+575.046344348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert") pod "llmisvc-controller-manager-7584bddd84-vvhxz" (UID: "61b5d9bd-868f-48f1-9a82-0729a06be2dd") : secret "llmisvc-webhook-server-cert" not found
Jul 27 17:38:26.468050 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.467870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t796c\" (UniqueName: \"kubernetes.io/projected/61b5d9bd-868f-48f1-9a82-0729a06be2dd-kube-api-access-t796c\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.476729 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.476705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t796c\" (UniqueName: \"kubernetes.io/projected/61b5d9bd-868f-48f1-9a82-0729a06be2dd-kube-api-access-t796c\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.544377 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.544343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.665860 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.665829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.815827 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.815747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.917025 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.916991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:38:26.970793 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.970763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:26.973233 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:26.973211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert\") pod \"llmisvc-controller-manager-7584bddd84-vvhxz\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") " pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:27.208921 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:27.208840    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:27.335174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:27.335142    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-7584bddd84-vvhxz"]
Jul 27 17:38:27.338177 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:38:27.338150    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod61b5d9bd_868f_48f1_9a82_0729a06be2dd.slice/crio-6de988ad22c7e2d80f085847fbd4361fc1e8b4bec05fddab4820dcf4a209416f WatchSource:0}: Error finding container 6de988ad22c7e2d80f085847fbd4361fc1e8b4bec05fddab4820dcf4a209416f: Status 404 returned error can't find the container with id 6de988ad22c7e2d80f085847fbd4361fc1e8b4bec05fddab4820dcf4a209416f
Jul 27 17:38:27.611198 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:27.611162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz" event={"ID":"61b5d9bd-868f-48f1-9a82-0729a06be2dd","Type":"ContainerStarted","Data":"6de988ad22c7e2d80f085847fbd4361fc1e8b4bec05fddab4820dcf4a209416f"}
Jul 27 17:38:31.633356 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:31.633320    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz" event={"ID":"61b5d9bd-868f-48f1-9a82-0729a06be2dd","Type":"ContainerStarted","Data":"f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18"}
Jul 27 17:38:31.633811 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:31.633440    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:38:31.649934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:31.649893    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz" podStartSLOduration=2.256828955 podStartE2EDuration="5.649881313s" totalImagesPullingTime="3.393052358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:38:31.648227584 +0000 UTC m=+579.726660007" watchObservedRunningTime="2026-07-27 17:38:31.649881313 +0000 UTC m=+579.728313736"
Jul 27 17:38:52.323518 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:52.323485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:38:52.324422 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:52.324399    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:38:52.324563 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:52.324402    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:38:52.325472 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:52.325452    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:38:52.329476 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:52.329458    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:38:52.330319 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:38:52.330301    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:39:02.640517 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:02.640485    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 17:39:08.547181 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:08.547146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:09.895382 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:09.895346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:10.499073 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:10.499031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:10.549301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:10.549272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:20.511194 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:20.511117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:21.209963 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:21.209925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:24.123497 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:24.123463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:24.224867 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:24.224838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:25.690343 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:25.690309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:26.511894 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:26.511859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:28.095726 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:28.095697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:28.481018 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:28.480983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:29.111617 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:29.111583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:29.210985 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:29.210951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:30.905847 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:30.905818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:31.006124 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:31.006091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:31.105856 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:31.105822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:32.566112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:32.566073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:33.431743 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:33.431708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:33.532970 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:33.532940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:35.560683 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:35.560653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:35.661503 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:35.661467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:35.761859 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:35.761829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:44.807071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:44.807033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:44.902518 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:44.902481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:54.616319 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:54.616285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:59.436007 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.435963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:59.448163 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.448129    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"]
Jul 27 17:39:59.451340 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.451324    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.453786 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.453760    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ch7jb\""
Jul 27 17:39:59.453902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.453793    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 17:39:59.453902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.453821    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 17:39:59.453902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.453825    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 17:39:59.453902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.453833    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 17:39:59.466035 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.466015    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"]
Jul 27 17:39:59.470630 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.470714 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f124ae20-ee26-472a-b3ae-65873226c9dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.470714 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.470797 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.470833 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drj6\" (UniqueName: \"kubernetes.io/projected/f124ae20-ee26-472a-b3ae-65873226c9dd-kube-api-access-8drj6\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.470890 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.470926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.470903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.536223 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.536191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:39:59.571375 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.571537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f124ae20-ee26-472a-b3ae-65873226c9dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.571537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.571537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.571537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drj6\" (UniqueName: \"kubernetes.io/projected/f124ae20-ee26-472a-b3ae-65873226c9dd-kube-api-access-8drj6\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.571537 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.571799 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.571541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.572179 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.572153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.573829 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.573799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.574006 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.573987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.574086 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.574066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f124ae20-ee26-472a-b3ae-65873226c9dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.574128 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.574078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f124ae20-ee26-472a-b3ae-65873226c9dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.579478 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.579456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f124ae20-ee26-472a-b3ae-65873226c9dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.579599 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.579581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drj6\" (UniqueName: \"kubernetes.io/projected/f124ae20-ee26-472a-b3ae-65873226c9dd-kube-api-access-8drj6\") pod \"istiod-openshift-gateway-5677ccb49b-8bfqq\" (UID: \"f124ae20-ee26-472a-b3ae-65873226c9dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.760098 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.760073    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:39:59.895675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.895649    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"]
Jul 27 17:39:59.898033 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:39:59.898006    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf124ae20_ee26_472a_b3ae_65873226c9dd.slice/crio-dbe843eb9360c7d23076371b64e93066eac513d11a1733900ac72438a229064a WatchSource:0}: Error finding container dbe843eb9360c7d23076371b64e93066eac513d11a1733900ac72438a229064a: Status 404 returned error can't find the container with id dbe843eb9360c7d23076371b64e93066eac513d11a1733900ac72438a229064a
Jul 27 17:39:59.899921 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:39:59.899902    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:40:00.007302 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:00.007267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq" event={"ID":"f124ae20-ee26-472a-b3ae-65873226c9dd","Type":"ContainerStarted","Data":"dbe843eb9360c7d23076371b64e93066eac513d11a1733900ac72438a229064a"}
Jul 27 17:40:00.350568 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:00.350530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:02.591026 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:02.590992    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 17:40:02.591273 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:02.591065    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 17:40:03.025449 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.025392    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq" event={"ID":"f124ae20-ee26-472a-b3ae-65873226c9dd","Type":"ContainerStarted","Data":"5baec778e7558fde81706f7d60b258a86014ac058861dee9c86a564851463da4"}
Jul 27 17:40:03.025638 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.025573    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:40:03.027558 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.027533    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-8bfqq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 17:40:03.027693 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.027583    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq" podUID="f124ae20-ee26-472a-b3ae-65873226c9dd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 17:40:03.046515 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.046454    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq" podStartSLOduration=1.355699457 podStartE2EDuration="4.046418053s" totalImagesPullingTime="2.690718596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:40:03.044064687 +0000 UTC m=+671.122497113" watchObservedRunningTime="2026-07-27 17:40:03.046418053 +0000 UTC m=+671.124850480"
Jul 27 17:40:03.118717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.118686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:03.373240 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.373115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:03.440452 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:03.440376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:04.030700 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:04.030676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bfqq"
Jul 27 17:40:06.011075 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:06.011029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:06.107517 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:06.107480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:10.125070 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:10.125037    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:11.530222 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:11.530182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:11.630929 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:11.630892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:17.009320 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:17.009274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:17.115349 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:17.115315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:24.754567 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:24.754525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:25.546476 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:25.546440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:26.361085 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:26.361052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:29.541630 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:29.541596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:29.957167 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:29.957081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:30.193983 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:30.193947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:30.301784 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:30.301751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:31.935853 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:31.935816    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-56fb6f6856-t4rz2"]
Jul 27 17:40:31.945230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:31.945205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:31.948247 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:31.948221    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8fpwj\""
Jul 27 17:40:31.948844 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:31.948820    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 17:40:31.949162 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:31.949145    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 17:40:31.949700 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:31.949679    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-56fb6f6856-t4rz2"]
Jul 27 17:40:32.066142 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.066102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw7fs\" (UniqueName: \"kubernetes.io/projected/781acdec-7b07-43ff-9905-46d7bec79e89-kube-api-access-tw7fs\") pod \"authorino-operator-56fb6f6856-t4rz2\" (UID: \"781acdec-7b07-43ff-9905-46d7bec79e89\") " pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:32.167541 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.167512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw7fs\" (UniqueName: \"kubernetes.io/projected/781acdec-7b07-43ff-9905-46d7bec79e89-kube-api-access-tw7fs\") pod \"authorino-operator-56fb6f6856-t4rz2\" (UID: \"781acdec-7b07-43ff-9905-46d7bec79e89\") " pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:32.182179 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.182148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw7fs\" (UniqueName: \"kubernetes.io/projected/781acdec-7b07-43ff-9905-46d7bec79e89-kube-api-access-tw7fs\") pod \"authorino-operator-56fb6f6856-t4rz2\" (UID: \"781acdec-7b07-43ff-9905-46d7bec79e89\") " pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:32.256214 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.256184    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:32.389159 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.389138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-56fb6f6856-t4rz2"]
Jul 27 17:40:32.391869 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:40:32.391840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781acdec_7b07_43ff_9905_46d7bec79e89.slice/crio-9a159c476585ffa67b80030be64ed368b84c7caa378a79c4d648f0b7a2faf18b WatchSource:0}: Error finding container 9a159c476585ffa67b80030be64ed368b84c7caa378a79c4d648f0b7a2faf18b: Status 404 returned error can't find the container with id 9a159c476585ffa67b80030be64ed368b84c7caa378a79c4d648f0b7a2faf18b
Jul 27 17:40:32.580857 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.580788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:32.681016 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:32.680984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:33.159934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:33.159902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2" event={"ID":"781acdec-7b07-43ff-9905-46d7bec79e89","Type":"ContainerStarted","Data":"9a159c476585ffa67b80030be64ed368b84c7caa378a79c4d648f0b7a2faf18b"}
Jul 27 17:40:33.418742 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:33.418663    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:33.521548 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:33.521515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:34.164694 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.164635    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"]
Jul 27 17:40:34.173029 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.173002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:34.175513 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.175491    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4jcmj\""
Jul 27 17:40:34.178992 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.178966    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"]
Jul 27 17:40:34.286575 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.286534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhgg\" (UniqueName: \"kubernetes.io/projected/d26d5fe0-b579-4098-bd73-2c2d76407a20-kube-api-access-bdhgg\") pod \"limitador-operator-controller-manager-774d94476f-4cd9s\" (UID: \"d26d5fe0-b579-4098-bd73-2c2d76407a20\") " pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:34.387365 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.387326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhgg\" (UniqueName: \"kubernetes.io/projected/d26d5fe0-b579-4098-bd73-2c2d76407a20-kube-api-access-bdhgg\") pod \"limitador-operator-controller-manager-774d94476f-4cd9s\" (UID: \"d26d5fe0-b579-4098-bd73-2c2d76407a20\") " pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:34.401025 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.400995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhgg\" (UniqueName: \"kubernetes.io/projected/d26d5fe0-b579-4098-bd73-2c2d76407a20-kube-api-access-bdhgg\") pod \"limitador-operator-controller-manager-774d94476f-4cd9s\" (UID: \"d26d5fe0-b579-4098-bd73-2c2d76407a20\") " pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:34.486881 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.486850    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:34.757462 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.757371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:34.985644 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:34.985532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"]
Jul 27 17:40:34.987838 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:40:34.987804    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd26d5fe0_b579_4098_bd73_2c2d76407a20.slice/crio-c1c5869b762be149121326526de02f9d32e687cc77d1bdea6ab7ebcfc5eb9c49 WatchSource:0}: Error finding container c1c5869b762be149121326526de02f9d32e687cc77d1bdea6ab7ebcfc5eb9c49: Status 404 returned error can't find the container with id c1c5869b762be149121326526de02f9d32e687cc77d1bdea6ab7ebcfc5eb9c49
Jul 27 17:40:35.179498 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:35.179372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s" event={"ID":"d26d5fe0-b579-4098-bd73-2c2d76407a20","Type":"ContainerStarted","Data":"c1c5869b762be149121326526de02f9d32e687cc77d1bdea6ab7ebcfc5eb9c49"}
Jul 27 17:40:35.180817 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:35.180798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2" event={"ID":"781acdec-7b07-43ff-9905-46d7bec79e89","Type":"ContainerStarted","Data":"4951626d74fa2fc167b460bde9915159157f25611e4dd5bda07f9d2cbe093979"}
Jul 27 17:40:35.180900 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:35.180854    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:35.198638 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:35.198590    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2" podStartSLOduration=1.669792561 podStartE2EDuration="4.19857612s" totalImagesPullingTime="2.528783559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:40:35.19786969 +0000 UTC m=+703.276302126" watchObservedRunningTime="2026-07-27 17:40:35.19857612 +0000 UTC m=+703.277008543"
Jul 27 17:40:35.387160 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:35.387122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:36.556473 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:36.556419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:36.655641 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:36.655610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:38.197765 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:38.197731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s" event={"ID":"d26d5fe0-b579-4098-bd73-2c2d76407a20","Type":"ContainerStarted","Data":"056d2b2823fb4e852f41a583f280644ec0d09c8206a7679d1199d61ae18282f5"}
Jul 27 17:40:38.198180 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:38.197794    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:38.236367 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:38.236314    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s" podStartSLOduration=1.2036630289999999 podStartE2EDuration="4.236298515s" totalImagesPullingTime="3.032635486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:40:38.234272367 +0000 UTC m=+706.312704789" watchObservedRunningTime="2026-07-27 17:40:38.236298515 +0000 UTC m=+706.314730938"
Jul 27 17:40:39.780514 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:39.780483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:39.879447 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:39.879397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:40.216751 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:40.216695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:40.357306 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:40.357270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:40.610278 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:40.610193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:40.709396 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:40.709359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:46.188087 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:46.188001    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-56fb6f6856-t4rz2"
Jul 27 17:40:48.970194 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:48.970148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:49.070332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:49.070285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:49.205155 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:49.205124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-774d94476f-4cd9s"
Jul 27 17:40:52.109845 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:52.109806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:52.214161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:52.214128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:54.454188 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:54.454152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:40:54.552423 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:40:54.552390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:01.407321 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:01.407290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:01.557758 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:01.557726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:01.794320 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:01.794289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:01.890744 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:01.890713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:01.990614 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:01.990580    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:02.067712 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:02.067639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:06.027979 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:06.027940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:06.101050 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:06.101014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:10.826293 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:10.826254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:10.933662 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:10.933627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:13.786044 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:13.786012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:13.886476 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:13.886446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:16.274208 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:16.274171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:16.370244 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:16.370213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:18.928019 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:18.927982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:19.023269 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.023229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:19.132389 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.132354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:19.286794 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.286763    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"]
Jul 27 17:41:19.289224 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.289200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.291859 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.291833    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 17:41:19.291962 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.291882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6qg96\""
Jul 27 17:41:19.301707 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.301681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"]
Jul 27 17:41:19.332215 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.332174    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"]
Jul 27 17:41:19.384789 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.384758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngnpv\" (UniqueName: \"kubernetes.io/projected/b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3-kube-api-access-ngnpv\") pod \"limitador-limitador-85d9db4d5d-fcvd7\" (UID: \"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.384949 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.384819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3-config-file\") pod \"limitador-limitador-85d9db4d5d-fcvd7\" (UID: \"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.485367 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.485333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngnpv\" (UniqueName: \"kubernetes.io/projected/b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3-kube-api-access-ngnpv\") pod \"limitador-limitador-85d9db4d5d-fcvd7\" (UID: \"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.485584 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.485378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3-config-file\") pod \"limitador-limitador-85d9db4d5d-fcvd7\" (UID: \"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.485965 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.485945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3-config-file\") pod \"limitador-limitador-85d9db4d5d-fcvd7\" (UID: \"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.494954 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.494927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngnpv\" (UniqueName: \"kubernetes.io/projected/b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3-kube-api-access-ngnpv\") pod \"limitador-limitador-85d9db4d5d-fcvd7\" (UID: \"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.599147 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.599074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:19.747508 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.747463    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"]
Jul 27 17:41:19.750656 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:41:19.750628    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ab4268_cbe2_4f53_b5c2_46fe7d7985a3.slice/crio-ef28de8b7d37b4b1c29051a3b8ebe1d04bd8bb13684518c570fb4e036a9c7377 WatchSource:0}: Error finding container ef28de8b7d37b4b1c29051a3b8ebe1d04bd8bb13684518c570fb4e036a9c7377: Status 404 returned error can't find the container with id ef28de8b7d37b4b1c29051a3b8ebe1d04bd8bb13684518c570fb4e036a9c7377
Jul 27 17:41:19.780030 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.780005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:19.862793 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:19.862724    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:20.387058 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:20.387024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7" event={"ID":"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3","Type":"ContainerStarted","Data":"ef28de8b7d37b4b1c29051a3b8ebe1d04bd8bb13684518c570fb4e036a9c7377"}
Jul 27 17:41:21.305698 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:21.305615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:21.393115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:21.393080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7" event={"ID":"b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3","Type":"ContainerStarted","Data":"1bd88c3e011e4cbb741476183cb6bf050da343c000ce3a41875a9bf6dd2a4d27"}
Jul 27 17:41:21.393568 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:21.393232    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:21.410963 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:21.410907    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7" podStartSLOduration=1.149201212 podStartE2EDuration="2.410894307s" totalImagesPullingTime="1.261693095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:41:21.410420595 +0000 UTC m=+749.488853019" watchObservedRunningTime="2026-07-27 17:41:21.410894307 +0000 UTC m=+749.489326730"
Jul 27 17:41:32.399328 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:32.399298    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-fcvd7"
Jul 27 17:41:38.729244 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:38.729210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:39.077205 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:39.077108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:50.346585 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:50.346553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:41:50.447782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:41:50.447738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:10.059382 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:10.059315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:10.343859 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:10.343777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:31.512356 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:31.512264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:31.607024 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:31.606980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:31.692796 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:31.692760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:31.793348 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:31.793252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:32.253877 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:32.253844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:32.354073 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:32.354042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:55.600531 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:55.600495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:42:55.693826 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:42:55.693793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:43:35.568912 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:35.568865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:43:35.625755 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:35.625721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:43:52.350733 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:52.350702    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:43:52.351713 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:52.351694    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:43:52.352483 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:52.352463    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:43:52.353344 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:52.353325    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:43:52.356480 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:52.356463    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:43:52.357917 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:43:52.357892    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:44:08.414617 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:08.414579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:08.513841 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:08.513809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:21.997183 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:21.997153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:24.489692 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:24.489662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:30.265476 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:30.265439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:30.403079 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:30.403054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:31.392215 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:31.392185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:31.494023 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:31.493990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:40.434896 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:40.434861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:44.304508 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.304470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:44.593609 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.593526    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"]
Jul 27 17:44:44.597150 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.597130    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.600724 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.600703    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Jul 27 17:44:44.600724 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.600717    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Jul 27 17:44:44.600877 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.600703    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Jul 27 17:44:44.600877 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.600759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-pvj4p\""
Jul 27 17:44:44.600877 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.600791    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-fbrps\""
Jul 27 17:44:44.606319 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.606292    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"]
Jul 27 17:44:44.619400 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.619377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:44:44.706751 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.706713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvbn7\" (UniqueName: \"kubernetes.io/projected/94276c72-045d-431e-8733-fa710f30f1a0-kube-api-access-mvbn7\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.706911 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.706765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94276c72-045d-431e-8733-fa710f30f1a0-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.807833 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.807800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94276c72-045d-431e-8733-fa710f30f1a0-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.807969 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.807929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvbn7\" (UniqueName: \"kubernetes.io/projected/94276c72-045d-431e-8733-fa710f30f1a0-kube-api-access-mvbn7\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.810292 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.810262    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94276c72-045d-431e-8733-fa710f30f1a0-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.815714 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.815694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvbn7\" (UniqueName: \"kubernetes.io/projected/94276c72-045d-431e-8733-fa710f30f1a0-kube-api-access-mvbn7\") pod \"fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:44.908027 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:44.907952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:45.039594 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:45.039571    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"]
Jul 27 17:44:45.041901 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:44:45.041878    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94276c72_045d_431e_8733_fa710f30f1a0.slice/crio-ce238210408e065615fc1abdc0e593d9ec994008848e0ab1e26e753f2337a87d WatchSource:0}: Error finding container ce238210408e065615fc1abdc0e593d9ec994008848e0ab1e26e753f2337a87d: Status 404 returned error can't find the container with id ce238210408e065615fc1abdc0e593d9ec994008848e0ab1e26e753f2337a87d
Jul 27 17:44:45.318096 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:45.318059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" event={"ID":"94276c72-045d-431e-8733-fa710f30f1a0","Type":"ContainerStarted","Data":"ce238210408e065615fc1abdc0e593d9ec994008848e0ab1e26e753f2337a87d"}
Jul 27 17:44:48.336646 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:48.336612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" event={"ID":"94276c72-045d-431e-8733-fa710f30f1a0","Type":"ContainerStarted","Data":"45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4"}
Jul 27 17:44:48.337008 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:48.336754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:44:48.352315 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:44:48.352264    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" podStartSLOduration=2.012239383 podStartE2EDuration="4.352246685s" totalImagesPullingTime="2.340007302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:44:48.351777771 +0000 UTC m=+956.430210192" watchObservedRunningTime="2026-07-27 17:44:48.352246685 +0000 UTC m=+956.430679109"
Jul 27 17:45:09.538298 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:09.538263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:19.343091 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:19.343010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:45:25.705654 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:25.705616    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"]
Jul 27 17:45:25.716402 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:25.716360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:25.991036 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:25.991002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:29.343752 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:29.343709    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" podUID="94276c72-045d-431e-8733-fa710f30f1a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:45:39.343908 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:39.343867    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" podUID="94276c72-045d-431e-8733-fa710f30f1a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:45:40.732840 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:40.732795    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" podUID="94276c72-045d-431e-8733-fa710f30f1a0" containerName="main" containerID="cri-o://45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4" gracePeriod=45
Jul 27 17:45:40.981452 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:40.981413    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:45:41.093966 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.093879    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvbn7\" (UniqueName: \"kubernetes.io/projected/94276c72-045d-431e-8733-fa710f30f1a0-kube-api-access-mvbn7\") pod \"94276c72-045d-431e-8733-fa710f30f1a0\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") "
Jul 27 17:45:41.094115 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.093991    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94276c72-045d-431e-8733-fa710f30f1a0-tls-certs\") pod \"94276c72-045d-431e-8733-fa710f30f1a0\" (UID: \"94276c72-045d-431e-8733-fa710f30f1a0\") "
Jul 27 17:45:41.096080 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.096055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94276c72-045d-431e-8733-fa710f30f1a0-kube-api-access-mvbn7" (OuterVolumeSpecName: "kube-api-access-mvbn7") pod "94276c72-045d-431e-8733-fa710f30f1a0" (UID: "94276c72-045d-431e-8733-fa710f30f1a0"). InnerVolumeSpecName "kube-api-access-mvbn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:45:41.096191 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.096135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94276c72-045d-431e-8733-fa710f30f1a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94276c72-045d-431e-8733-fa710f30f1a0" (UID: "94276c72-045d-431e-8733-fa710f30f1a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:45:41.195239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.195204    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94276c72-045d-431e-8733-fa710f30f1a0-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:45:41.195239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.195233    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvbn7\" (UniqueName: \"kubernetes.io/projected/94276c72-045d-431e-8733-fa710f30f1a0-kube-api-access-mvbn7\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:45:41.591373 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.591339    2577 generic.go:356] "Generic (PLEG): container finished" podID="94276c72-045d-431e-8733-fa710f30f1a0" containerID="45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4" exitCode=0
Jul 27 17:45:41.591553 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.591385    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"
Jul 27 17:45:41.591553 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.591421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" event={"ID":"94276c72-045d-431e-8733-fa710f30f1a0","Type":"ContainerDied","Data":"45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4"}
Jul 27 17:45:41.591553 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.591487    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n" event={"ID":"94276c72-045d-431e-8733-fa710f30f1a0","Type":"ContainerDied","Data":"ce238210408e065615fc1abdc0e593d9ec994008848e0ab1e26e753f2337a87d"}
Jul 27 17:45:41.591553 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.591509    2577 scope.go:117] "RemoveContainer" containerID="45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4"
Jul 27 17:45:41.601213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.601193    2577 scope.go:117] "RemoveContainer" containerID="45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4"
Jul 27 17:45:41.601497 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:45:41.601478    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4\": container with ID starting with 45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4 not found: ID does not exist" containerID="45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4"
Jul 27 17:45:41.601583 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.601505    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4"} err="failed to get container status \"45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4\": rpc error: code = NotFound desc = could not find container \"45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4\": container with ID starting with 45e9a41b24e3cc87cc95aaf185abe7e18256577ed63199500cf788b09fc80fd4 not found: ID does not exist"
Jul 27 17:45:41.612154 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.612129    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"]
Jul 27 17:45:41.616332 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:41.616309    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-5cfd47986-r9h6n"]
Jul 27 17:45:42.485036 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:42.484999    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94276c72-045d-431e-8733-fa710f30f1a0" path="/var/lib/kubelet/pods/94276c72-045d-431e-8733-fa710f30f1a0/volumes"
Jul 27 17:45:45.613384 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:45.613340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:45.707168 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:45.707125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:51.592092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:51.592060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:57.140134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.140089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:57.207564 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.207529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:57.820349 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.820308    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:57.998336 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.998299    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"]
Jul 27 17:45:57.998764 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.998739    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94276c72-045d-431e-8733-fa710f30f1a0" containerName="main"
Jul 27 17:45:57.998764 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.998765    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="94276c72-045d-431e-8733-fa710f30f1a0" containerName="main"
Jul 27 17:45:57.998993 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:57.998867    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="94276c72-045d-431e-8733-fa710f30f1a0" containerName="main"
Jul 27 17:45:58.001883 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.001865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.005954 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.005931    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-epp-sa-dockercfg-5ht5p\""
Jul 27 17:45:58.006059 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.006019    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Jul 27 17:45:58.006059 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.006035    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Jul 27 17:45:58.006215 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.006200    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Jul 27 17:45:58.006347 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.006333    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-7jz5v\""
Jul 27 17:45:58.016217 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.016089    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"]
Jul 27 17:45:58.026788 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.026759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:45:58.143520 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.143409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrfh\" (UniqueName: \"kubernetes.io/projected/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-kube-api-access-khrfh\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.143520 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.143477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.244525 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.244486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrfh\" (UniqueName: \"kubernetes.io/projected/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-kube-api-access-khrfh\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.244525 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.244533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.247043 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.247019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.252665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.252641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrfh\" (UniqueName: \"kubernetes.io/projected/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-kube-api-access-khrfh\") pod \"fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.313593 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.313553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.446519 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.446495    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"]
Jul 27 17:45:58.448487 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:45:58.448456    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4217ad0b_eaef_496e_97c9_f7bb0d1d4a30.slice/crio-a0bde345fddb87fed9cbbc25b6406bbf28a9db8e1596b8dd0dac087f8efe4ea3 WatchSource:0}: Error finding container a0bde345fddb87fed9cbbc25b6406bbf28a9db8e1596b8dd0dac087f8efe4ea3: Status 404 returned error can't find the container with id a0bde345fddb87fed9cbbc25b6406bbf28a9db8e1596b8dd0dac087f8efe4ea3
Jul 27 17:45:58.673853 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.673757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" event={"ID":"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30","Type":"ContainerStarted","Data":"c604714b7d4a20be1ede3765370916b8d7074a3e75ed3deb77ad3de7416f51a2"}
Jul 27 17:45:58.673853 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.673835    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" event={"ID":"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30","Type":"ContainerStarted","Data":"a0bde345fddb87fed9cbbc25b6406bbf28a9db8e1596b8dd0dac087f8efe4ea3"}
Jul 27 17:45:58.674030 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.673873    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:45:58.692211 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:45:58.692159    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" podStartSLOduration=1.6921419 podStartE2EDuration="1.6921419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:45:58.690401147 +0000 UTC m=+1026.768833570" watchObservedRunningTime="2026-07-27 17:45:58.6921419 +0000 UTC m=+1026.770574322"
Jul 27 17:46:08.778502 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:08.778472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:17.057447 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:17.057394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:29.680382 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:29.680355    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:46:35.715748 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:35.715719    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"]
Jul 27 17:46:35.719571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:35.719541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:35.782728 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:35.782694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:39.680190 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:39.680153    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:46:49.680662 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:49.680625    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:46:50.736376 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:50.736300    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerName="main" containerID="cri-o://c604714b7d4a20be1ede3765370916b8d7074a3e75ed3deb77ad3de7416f51a2" gracePeriod=45
Jul 27 17:46:50.916091 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:50.916062    2577 generic.go:356] "Generic (PLEG): container finished" podID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerID="c604714b7d4a20be1ede3765370916b8d7074a3e75ed3deb77ad3de7416f51a2" exitCode=0
Jul 27 17:46:50.916229 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:50.916130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" event={"ID":"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30","Type":"ContainerDied","Data":"c604714b7d4a20be1ede3765370916b8d7074a3e75ed3deb77ad3de7416f51a2"}
Jul 27 17:46:50.996328 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:50.996274    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:46:51.102940 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.102909    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khrfh\" (UniqueName: \"kubernetes.io/projected/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-kube-api-access-khrfh\") pod \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") "
Jul 27 17:46:51.103087 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.102957    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-tls-certs\") pod \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\" (UID: \"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30\") "
Jul 27 17:46:51.105070 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.105038    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" (UID: "4217ad0b-eaef-496e-97c9-f7bb0d1d4a30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:46:51.105070 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.105052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-kube-api-access-khrfh" (OuterVolumeSpecName: "kube-api-access-khrfh") pod "4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" (UID: "4217ad0b-eaef-496e-97c9-f7bb0d1d4a30"). InnerVolumeSpecName "kube-api-access-khrfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:46:51.203698 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.203650    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:46:51.203698 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.203692    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khrfh\" (UniqueName: \"kubernetes.io/projected/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30-kube-api-access-khrfh\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:46:51.922056 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.922020    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"
Jul 27 17:46:51.922557 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.922053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6" event={"ID":"4217ad0b-eaef-496e-97c9-f7bb0d1d4a30","Type":"ContainerDied","Data":"a0bde345fddb87fed9cbbc25b6406bbf28a9db8e1596b8dd0dac087f8efe4ea3"}
Jul 27 17:46:51.922557 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.922119    2577 scope.go:117] "RemoveContainer" containerID="c604714b7d4a20be1ede3765370916b8d7074a3e75ed3deb77ad3de7416f51a2"
Jul 27 17:46:51.943045 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.943022    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"]
Jul 27 17:46:51.946458 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:51.946421    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-69bbd6f4df-lr2k6"]
Jul 27 17:46:52.484753 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:52.484717    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" path="/var/lib/kubelet/pods/4217ad0b-eaef-496e-97c9-f7bb0d1d4a30/volumes"
Jul 27 17:46:55.751338 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:55.751298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:55.850943 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:55.850905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:55.964606 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:55.964559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:57.607122 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:57.607081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:57.680699 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:57.680666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:46:59.645308 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:46:59.645266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:01.409585 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:01.409542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:02.592861 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:02.592805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:03.609304 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:03.609265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:03.719173 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:03.719142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:03.825395 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:03.825360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:03.904841 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:03.904760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:03.977297 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:03.977253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:07.114423 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:07.114375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:08.691073 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:08.691041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:08.791466 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:08.791419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:08.891857 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:08.891824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:14.489158 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:14.489128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:15.044558 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:15.044522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:20.954906 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:20.954873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:23.541441 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:23.541391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:25.599362 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:25.599329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:25.693254 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:25.693218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:25.915956 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:25.915864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:30.159670 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:30.159633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:30.229611 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:30.229579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:33.461568 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.461483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:33.682870 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.682832    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"]
Jul 27 17:47:33.683247 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.683231    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerName="main"
Jul 27 17:47:33.683333 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.683250    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerName="main"
Jul 27 17:47:33.683387 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.683358    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4217ad0b-eaef-496e-97c9-f7bb0d1d4a30" containerName="main"
Jul 27 17:47:33.686236 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.686217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:33.690132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.690113    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Jul 27 17:47:33.690364 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.690338    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Jul 27 17:47:33.690486 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.690468    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-srq75\""
Jul 27 17:47:33.690571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.690482    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Jul 27 17:47:33.690571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.690482    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-hgx8z\""
Jul 27 17:47:33.698203 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.697858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"]
Jul 27 17:47:33.703618 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.703589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:33.784993 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.784960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59lqf\" (UniqueName: \"kubernetes.io/projected/b4b7806f-a702-4db7-a8bb-2100a99542ec-kube-api-access-59lqf\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:33.785132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.784998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b7806f-a702-4db7-a8bb-2100a99542ec-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:33.886138 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.886102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59lqf\" (UniqueName: \"kubernetes.io/projected/b4b7806f-a702-4db7-a8bb-2100a99542ec-kube-api-access-59lqf\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:33.886138 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.886141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b7806f-a702-4db7-a8bb-2100a99542ec-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:33.888596 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.888575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b7806f-a702-4db7-a8bb-2100a99542ec-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:33.895441 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:33.895402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59lqf\" (UniqueName: \"kubernetes.io/projected/b4b7806f-a702-4db7-a8bb-2100a99542ec-kube-api-access-59lqf\") pod \"auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:34.002755 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:34.002715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:34.135542 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:34.135512    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"]
Jul 27 17:47:34.138913 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:47:34.138886    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b7806f_a702_4db7_a8bb_2100a99542ec.slice/crio-6974c7ae87ab18bbf531d7c26b62804e69b37b275bc8f1a7fa8e2843e59b4705 WatchSource:0}: Error finding container 6974c7ae87ab18bbf531d7c26b62804e69b37b275bc8f1a7fa8e2843e59b4705: Status 404 returned error can't find the container with id 6974c7ae87ab18bbf531d7c26b62804e69b37b275bc8f1a7fa8e2843e59b4705
Jul 27 17:47:35.128844 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:35.128814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" event={"ID":"b4b7806f-a702-4db7-a8bb-2100a99542ec","Type":"ContainerStarted","Data":"1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8"}
Jul 27 17:47:35.129214 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:35.128860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" event={"ID":"b4b7806f-a702-4db7-a8bb-2100a99542ec","Type":"ContainerStarted","Data":"6974c7ae87ab18bbf531d7c26b62804e69b37b275bc8f1a7fa8e2843e59b4705"}
Jul 27 17:47:35.129214 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:35.128895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:47:35.149489 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:35.149423    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" podStartSLOduration=2.149409355 podStartE2EDuration="2.149409355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:47:35.147423157 +0000 UTC m=+1123.225855581" watchObservedRunningTime="2026-07-27 17:47:35.149409355 +0000 UTC m=+1123.227841778"
Jul 27 17:47:36.201197 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:36.201167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:36.799498 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:36.799400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:42.651047 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:42.651016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:43.094574 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:43.094539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:43.190194 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:43.190164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:44.835040 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:44.834997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:49.670310 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:49.670268    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:51.416648 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:51.416613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:51.486280 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:51.486244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:55.515819 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:55.515786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:47:55.622954 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:47:55.622914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:03.396097 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:03.396062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:03.543009 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:03.542950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:03.669844 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:03.669759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:03.917876 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:03.917840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:06.135417 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:06.135389    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:48:12.475626 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:12.475591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:22.764127 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:22.764048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:52.378778 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:52.378751    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:48:52.379618 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:52.379598    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:48:52.380898 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:52.380881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:48:52.381734 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:52.381712    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:48:52.384603 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:52.384579    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:48:52.386534 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:52.386518    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:48:59.088696 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:59.088661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:48:59.188362 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:48:59.188323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:49:22.265631 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:49:22.265594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:35.352275 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:35.352240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:47.989130 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:47.989090    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"]
Jul 27 17:50:48.003777 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:48.003744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:48.067043 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:48.067015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:53.280633 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:53.280600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:53.373492 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:53.373459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:50:56.135357 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:50:56.135322    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" podUID="b4b7806f-a702-4db7-a8bb-2100a99542ec" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:51:03.010937 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.010884    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" podUID="b4b7806f-a702-4db7-a8bb-2100a99542ec" containerName="main" containerID="cri-o://1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8" gracePeriod=45
Jul 27 17:51:03.264792 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.264721    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:51:03.399930 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.399892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b7806f-a702-4db7-a8bb-2100a99542ec-tls-certs\") pod \"b4b7806f-a702-4db7-a8bb-2100a99542ec\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") "
Jul 27 17:51:03.400125 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.399962    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59lqf\" (UniqueName: \"kubernetes.io/projected/b4b7806f-a702-4db7-a8bb-2100a99542ec-kube-api-access-59lqf\") pod \"b4b7806f-a702-4db7-a8bb-2100a99542ec\" (UID: \"b4b7806f-a702-4db7-a8bb-2100a99542ec\") "
Jul 27 17:51:03.402213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.402177    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b7806f-a702-4db7-a8bb-2100a99542ec-kube-api-access-59lqf" (OuterVolumeSpecName: "kube-api-access-59lqf") pod "b4b7806f-a702-4db7-a8bb-2100a99542ec" (UID: "b4b7806f-a702-4db7-a8bb-2100a99542ec"). InnerVolumeSpecName "kube-api-access-59lqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:51:03.402213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.402201    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4b7806f-a702-4db7-a8bb-2100a99542ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4b7806f-a702-4db7-a8bb-2100a99542ec" (UID: "b4b7806f-a702-4db7-a8bb-2100a99542ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:51:03.501453 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.501386    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4b7806f-a702-4db7-a8bb-2100a99542ec-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:51:03.501453 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:03.501451    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59lqf\" (UniqueName: \"kubernetes.io/projected/b4b7806f-a702-4db7-a8bb-2100a99542ec-kube-api-access-59lqf\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:51:04.097618 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.097587    2577 generic.go:356] "Generic (PLEG): container finished" podID="b4b7806f-a702-4db7-a8bb-2100a99542ec" containerID="1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8" exitCode=0
Jul 27 17:51:04.098074 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.097636    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"
Jul 27 17:51:04.098074 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.097667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" event={"ID":"b4b7806f-a702-4db7-a8bb-2100a99542ec","Type":"ContainerDied","Data":"1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8"}
Jul 27 17:51:04.098074 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.097721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47" event={"ID":"b4b7806f-a702-4db7-a8bb-2100a99542ec","Type":"ContainerDied","Data":"6974c7ae87ab18bbf531d7c26b62804e69b37b275bc8f1a7fa8e2843e59b4705"}
Jul 27 17:51:04.098074 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.097745    2577 scope.go:117] "RemoveContainer" containerID="1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8"
Jul 27 17:51:04.106396 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.106377    2577 scope.go:117] "RemoveContainer" containerID="1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8"
Jul 27 17:51:04.106711 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:51:04.106690    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8\": container with ID starting with 1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8 not found: ID does not exist" containerID="1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8"
Jul 27 17:51:04.106791 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.106724    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8"} err="failed to get container status \"1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8\": rpc error: code = NotFound desc = could not find container \"1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8\": container with ID starting with 1072cbb156f4d5ebb1c485bc42ce5f17619680c46e11620887cdfd2c5657ddf8 not found: ID does not exist"
Jul 27 17:51:04.120362 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.120332    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"]
Jul 27 17:51:04.127620 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.127595    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-5dbb5856d8-qht47"]
Jul 27 17:51:04.482569 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:04.482537    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4b7806f-a702-4db7-a8bb-2100a99542ec" path="/var/lib/kubelet/pods/b4b7806f-a702-4db7-a8bb-2100a99542ec/volumes"
Jul 27 17:51:08.050087 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:08.050026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:08.215887 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:08.215854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:14.156054 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.155955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:14.530207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.527053    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"]
Jul 27 17:51:14.530207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.528178    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4b7806f-a702-4db7-a8bb-2100a99542ec" containerName="main"
Jul 27 17:51:14.530207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.528201    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4b7806f-a702-4db7-a8bb-2100a99542ec" containerName="main"
Jul 27 17:51:14.530207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.528390    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4b7806f-a702-4db7-a8bb-2100a99542ec" containerName="main"
Jul 27 17:51:14.533799 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.533772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.537396 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.537376    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-xh999\""
Jul 27 17:51:14.538620 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.538523    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-epp-sa-dockercfg-6d4db\""
Jul 27 17:51:14.538620 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.538596    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Jul 27 17:51:14.538785 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.538675    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\""
Jul 27 17:51:14.538785 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.538755    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\""
Jul 27 17:51:14.546877 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.546853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"]
Jul 27 17:51:14.553400 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.553377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:14.600400 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.600368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4379275-bf99-44a2-a4b8-0cfbe5c08799-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.600580 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.600491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d7nw\" (UniqueName: \"kubernetes.io/projected/e4379275-bf99-44a2-a4b8-0cfbe5c08799-kube-api-access-6d7nw\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.701782 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.701741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d7nw\" (UniqueName: \"kubernetes.io/projected/e4379275-bf99-44a2-a4b8-0cfbe5c08799-kube-api-access-6d7nw\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.701944 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.701828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4379275-bf99-44a2-a4b8-0cfbe5c08799-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.704275 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.704252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4379275-bf99-44a2-a4b8-0cfbe5c08799-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.709931 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.709906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d7nw\" (UniqueName: \"kubernetes.io/projected/e4379275-bf99-44a2-a4b8-0cfbe5c08799-kube-api-access-6d7nw\") pod \"auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.845555 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.845475    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:14.987110 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:14.987072    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"]
Jul 27 17:51:14.990221 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:51:14.990189    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4379275_bf99_44a2_a4b8_0cfbe5c08799.slice/crio-4c491704dae6f9469e4225560ce2825edf231878aab1339ea8d4647514efa154 WatchSource:0}: Error finding container 4c491704dae6f9469e4225560ce2825edf231878aab1339ea8d4647514efa154: Status 404 returned error can't find the container with id 4c491704dae6f9469e4225560ce2825edf231878aab1339ea8d4647514efa154
Jul 27 17:51:15.151457 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:15.151363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" event={"ID":"e4379275-bf99-44a2-a4b8-0cfbe5c08799","Type":"ContainerStarted","Data":"3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583"}
Jul 27 17:51:15.151457 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:15.151415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" event={"ID":"e4379275-bf99-44a2-a4b8-0cfbe5c08799","Type":"ContainerStarted","Data":"4c491704dae6f9469e4225560ce2825edf231878aab1339ea8d4647514efa154"}
Jul 27 17:51:15.151650 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:15.151600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:51:15.167660 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:15.167602    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" podStartSLOduration=1.1675830839999999 podStartE2EDuration="1.167583084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:51:15.167226438 +0000 UTC m=+1343.245658862" watchObservedRunningTime="2026-07-27 17:51:15.167583084 +0000 UTC m=+1343.246015508"
Jul 27 17:51:15.649173 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:15.649120    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:21.685212 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:21.685178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:21.782357 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:21.782315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:28.383637 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:28.383604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:28.475848 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:28.475817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:34.954112 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:34.954077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:40.469254 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:40.469216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:40.656887 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:40.656858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:51:46.157977 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:51:46.157944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:52:14.025002 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:52:14.024968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:21.863488 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:52:21.863447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:21.962664 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:52:21.962631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:30.849982 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:52:30.849949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:54.107704 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:52:54.107668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:52:54.213335 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:52:54.213302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:13.760251 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:13.760215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:13.850893 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:13.850859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:20.761661 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:20.761618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:24.922506 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:24.922469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:25.165673 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:25.165639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:27.504685 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:27.504645    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:27.602935 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:27.602901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:33.138334 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:33.138294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:34.792891 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:34.792857    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"]
Jul 27 17:53:34.800174 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:34.800139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:34.879534 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:34.879500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:36.157675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:36.157638    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:53:46.157385 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:46.157348    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:53:49.812165 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:49.812126    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerName="main" containerID="cri-o://3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583" gracePeriod=45
Jul 27 17:53:50.067884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.067826    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:53:50.162862 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.162827    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4379275-bf99-44a2-a4b8-0cfbe5c08799-tls-certs\") pod \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") "
Jul 27 17:53:50.163023 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.162880    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d7nw\" (UniqueName: \"kubernetes.io/projected/e4379275-bf99-44a2-a4b8-0cfbe5c08799-kube-api-access-6d7nw\") pod \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\" (UID: \"e4379275-bf99-44a2-a4b8-0cfbe5c08799\") "
Jul 27 17:53:50.165039 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.165005    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4379275-bf99-44a2-a4b8-0cfbe5c08799-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4379275-bf99-44a2-a4b8-0cfbe5c08799" (UID: "e4379275-bf99-44a2-a4b8-0cfbe5c08799"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:53:50.165039 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.165018    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4379275-bf99-44a2-a4b8-0cfbe5c08799-kube-api-access-6d7nw" (OuterVolumeSpecName: "kube-api-access-6d7nw") pod "e4379275-bf99-44a2-a4b8-0cfbe5c08799" (UID: "e4379275-bf99-44a2-a4b8-0cfbe5c08799"). InnerVolumeSpecName "kube-api-access-6d7nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:53:50.263558 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.263522    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4379275-bf99-44a2-a4b8-0cfbe5c08799-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:53:50.263558 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.263553    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d7nw\" (UniqueName: \"kubernetes.io/projected/e4379275-bf99-44a2-a4b8-0cfbe5c08799-kube-api-access-6d7nw\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:53:50.892967 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.892879    2577 generic.go:356] "Generic (PLEG): container finished" podID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerID="3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583" exitCode=0
Jul 27 17:53:50.892967 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.892928    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"
Jul 27 17:53:50.892967 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.892945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" event={"ID":"e4379275-bf99-44a2-a4b8-0cfbe5c08799","Type":"ContainerDied","Data":"3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583"}
Jul 27 17:53:50.893529 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.892997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj" event={"ID":"e4379275-bf99-44a2-a4b8-0cfbe5c08799","Type":"ContainerDied","Data":"4c491704dae6f9469e4225560ce2825edf231878aab1339ea8d4647514efa154"}
Jul 27 17:53:50.893529 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.893026    2577 scope.go:117] "RemoveContainer" containerID="3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583"
Jul 27 17:53:50.903740 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.903717    2577 scope.go:117] "RemoveContainer" containerID="3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583"
Jul 27 17:53:50.904036 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:53:50.904004    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583\": container with ID starting with 3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583 not found: ID does not exist" containerID="3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583"
Jul 27 17:53:50.904140 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.904034    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583"} err="failed to get container status \"3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583\": rpc error: code = NotFound desc = could not find container \"3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583\": container with ID starting with 3dc47b9e1b9d4c69d0e092e0818e9b4afcbe28a84b835ba0ae5d60bed3ef9583 not found: ID does not exist"
Jul 27 17:53:50.909401 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.909375    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"]
Jul 27 17:53:50.913835 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:50.913814    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-router-scheduler-6d845bb7dcvrqqj"]
Jul 27 17:53:52.406572 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.406545    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:53:52.407800 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.407779    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:53:52.409703 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.409682    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:53:52.410580 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.410558    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:53:52.412522 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.412490    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:53:52.415187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.415168    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:53:52.482665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:52.482640    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" path="/var/lib/kubelet/pods/e4379275-bf99-44a2-a4b8-0cfbe5c08799/volumes"
Jul 27 17:53:53.958999 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:53.958967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:53:54.062363 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:53:54.062322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:00.450241 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:00.450205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:03.743578 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.743546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:03.917002 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.916966    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"]
Jul 27 17:54:03.917313 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.917301    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerName="main"
Jul 27 17:54:03.917371 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.917315    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerName="main"
Jul 27 17:54:03.917408 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.917398    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4379275-bf99-44a2-a4b8-0cfbe5c08799" containerName="main"
Jul 27 17:54:03.921574 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.921552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:03.925442 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.925401    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-epp-sa-dockercfg-fwm6v\""
Jul 27 17:54:03.925563 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.925443    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"kube-root-ca.crt\""
Jul 27 17:54:03.925614 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.925562    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"default-dockercfg-ms69c\""
Jul 27 17:54:03.925684 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.925666    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"openshift-service-ca.crt\""
Jul 27 17:54:03.925775 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.925759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-disabled-no-token-required-396e4821\"/\"auth-disabled-test-kserve-self-signed-certs\""
Jul 27 17:54:03.929057 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.928836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"]
Jul 27 17:54:03.940761 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.940734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:03.984763 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.984726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksnxn\" (UniqueName: \"kubernetes.io/projected/954dff79-edb4-406d-b1b3-eb91a76e8863-kube-api-access-ksnxn\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:03.984898 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:03.984805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/954dff79-edb4-406d-b1b3-eb91a76e8863-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.086181 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.086100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/954dff79-edb4-406d-b1b3-eb91a76e8863-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.086322 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.086204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksnxn\" (UniqueName: \"kubernetes.io/projected/954dff79-edb4-406d-b1b3-eb91a76e8863-kube-api-access-ksnxn\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.088728 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.088703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/954dff79-edb4-406d-b1b3-eb91a76e8863-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.094360 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.094326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksnxn\" (UniqueName: \"kubernetes.io/projected/954dff79-edb4-406d-b1b3-eb91a76e8863-kube-api-access-ksnxn\") pod \"auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") " pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.233762 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.233723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.373104 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.372980    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"]
Jul 27 17:54:04.375494 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:54:04.375465    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod954dff79_edb4_406d_b1b3_eb91a76e8863.slice/crio-e22c598c4662d27026b6fe8921d02a53b89d7f865d91f6487f008fda9f38c6b9 WatchSource:0}: Error finding container e22c598c4662d27026b6fe8921d02a53b89d7f865d91f6487f008fda9f38c6b9: Status 404 returned error can't find the container with id e22c598c4662d27026b6fe8921d02a53b89d7f865d91f6487f008fda9f38c6b9
Jul 27 17:54:04.970264 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.970233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" event={"ID":"954dff79-edb4-406d-b1b3-eb91a76e8863","Type":"ContainerStarted","Data":"8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72"}
Jul 27 17:54:04.970658 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.970282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" event={"ID":"954dff79-edb4-406d-b1b3-eb91a76e8863","Type":"ContainerStarted","Data":"e22c598c4662d27026b6fe8921d02a53b89d7f865d91f6487f008fda9f38c6b9"}
Jul 27 17:54:04.970658 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.970353    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:04.988031 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:04.987984    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" podStartSLOduration=1.987969407 podStartE2EDuration="1.987969407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:54:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:54:04.985735042 +0000 UTC m=+1513.064167467" watchObservedRunningTime="2026-07-27 17:54:04.987969407 +0000 UTC m=+1513.066401831"
Jul 27 17:54:06.813880 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:06.813848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:06.888967 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:06.888931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:09.527205 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:09.527166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:19.319164 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:19.319125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:20.664571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:20.664532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:20.759295 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:20.759260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:35.977238 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:35.977205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:54:37.461814 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:37.461779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:37.560515 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:37.560479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:40.981838 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:40.981802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:41.232779 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:41.232688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:44.704291 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:44.704256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:50.938583 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:50.938546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:51.005028 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:51.004994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:57.583133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:57.583074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:54:58.282687 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:54:58.282647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:14.542251 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:14.542213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:14.697763 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:14.697732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:34.348806 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:34.348774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:34.449299 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:34.449270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:34.552484 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:34.552451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:40.559469 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:40.559416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:42.841688 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:42.841654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:43.033076 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:43.033022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:46.662896 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:46.662856    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:46.763180 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:46.763146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:55:46.904723 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:55:46.904688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:08.523950 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:08.523917    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"]
Jul 27 17:56:08.537687 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:08.537655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:08.593335 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:08.593303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:15.977562 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:15.977524    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" podUID="954dff79-edb4-406d-b1b3-eb91a76e8863" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:56:16.096230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:16.096200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:16.187569 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:16.187538    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:23.545378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:23.545329    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" podUID="954dff79-edb4-406d-b1b3-eb91a76e8863" containerName="main" containerID="cri-o://8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72" gracePeriod=45
Jul 27 17:56:23.796727 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:23.796654    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:56:23.921717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:23.921678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksnxn\" (UniqueName: \"kubernetes.io/projected/954dff79-edb4-406d-b1b3-eb91a76e8863-kube-api-access-ksnxn\") pod \"954dff79-edb4-406d-b1b3-eb91a76e8863\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") "
Jul 27 17:56:23.921905 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:23.921756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/954dff79-edb4-406d-b1b3-eb91a76e8863-tls-certs\") pod \"954dff79-edb4-406d-b1b3-eb91a76e8863\" (UID: \"954dff79-edb4-406d-b1b3-eb91a76e8863\") "
Jul 27 17:56:23.923861 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:23.923834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/954dff79-edb4-406d-b1b3-eb91a76e8863-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "954dff79-edb4-406d-b1b3-eb91a76e8863" (UID: "954dff79-edb4-406d-b1b3-eb91a76e8863"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:56:23.923963 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:23.923872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/954dff79-edb4-406d-b1b3-eb91a76e8863-kube-api-access-ksnxn" (OuterVolumeSpecName: "kube-api-access-ksnxn") pod "954dff79-edb4-406d-b1b3-eb91a76e8863" (UID: "954dff79-edb4-406d-b1b3-eb91a76e8863"). InnerVolumeSpecName "kube-api-access-ksnxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:56:24.023129 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.023098    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksnxn\" (UniqueName: \"kubernetes.io/projected/954dff79-edb4-406d-b1b3-eb91a76e8863-kube-api-access-ksnxn\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:56:24.023129 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.023124    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/954dff79-edb4-406d-b1b3-eb91a76e8863-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:56:24.617789 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.617752    2577 generic.go:356] "Generic (PLEG): container finished" podID="954dff79-edb4-406d-b1b3-eb91a76e8863" containerID="8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72" exitCode=0
Jul 27 17:56:24.618262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.617785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" event={"ID":"954dff79-edb4-406d-b1b3-eb91a76e8863","Type":"ContainerDied","Data":"8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72"}
Jul 27 17:56:24.618262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.617815    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"
Jul 27 17:56:24.618262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.617852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq" event={"ID":"954dff79-edb4-406d-b1b3-eb91a76e8863","Type":"ContainerDied","Data":"e22c598c4662d27026b6fe8921d02a53b89d7f865d91f6487f008fda9f38c6b9"}
Jul 27 17:56:24.618262 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.617878    2577 scope.go:117] "RemoveContainer" containerID="8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72"
Jul 27 17:56:24.626260 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.626246    2577 scope.go:117] "RemoveContainer" containerID="8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72"
Jul 27 17:56:24.626528 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:56:24.626502    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72\": container with ID starting with 8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72 not found: ID does not exist" containerID="8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72"
Jul 27 17:56:24.626597 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.626536    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72"} err="failed to get container status \"8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72\": rpc error: code = NotFound desc = could not find container \"8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72\": container with ID starting with 8cae7aedd0317d359edef215223002f92bdb1fbbd5677eb3c61d57abd03eaf72 not found: ID does not exist"
Jul 27 17:56:24.635599 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.635510    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"]
Jul 27 17:56:24.637711 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:24.637689    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-disabled-no-token-required-396e4821/auth-disabled-test-kserve-router-scheduler-7ccdc4c597-pvwlq"]
Jul 27 17:56:26.483579 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:26.483544    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="954dff79-edb4-406d-b1b3-eb91a76e8863" path="/var/lib/kubelet/pods/954dff79-edb4-406d-b1b3-eb91a76e8863/volumes"
Jul 27 17:56:28.610777 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:28.610736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:28.709148 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:28.709115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:34.865594 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:34.865557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:36.161930 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:36.161899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:36.258034 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:36.257992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:40.529399 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:40.529363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:40.600114 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:40.600079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:42.372110 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:42.372077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:48.228629 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:48.228587    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:48.303004 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:48.302969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:48.403454 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:48.403401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:56:48.556085 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:56:48.556008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:08.984323 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.984286    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"]
Jul 27 17:57:08.985213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.985189    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="954dff79-edb4-406d-b1b3-eb91a76e8863" containerName="main"
Jul 27 17:57:08.985213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.985212    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="954dff79-edb4-406d-b1b3-eb91a76e8863" containerName="main"
Jul 27 17:57:08.985361 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.985284    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="954dff79-edb4-406d-b1b3-eb91a76e8863" containerName="main"
Jul 27 17:57:08.988188 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.988173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:08.990879 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.990858    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"default-dockercfg-4khsr\""
Jul 27 17:57:08.990976 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.990865    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"openshift-service-ca.crt\""
Jul 27 17:57:08.991965 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.991947    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"kube-root-ca.crt\""
Jul 27 17:57:08.992027 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.991986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Jul 27 17:57:08.995557 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:08.995534    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"]
Jul 27 17:57:09.130715 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.130676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-model-cache\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.130926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.130742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tls-certs\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.130926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.130834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znh2n\" (UniqueName: \"kubernetes.io/projected/c620025a-5b51-4c8e-ad60-f0e6452a34d4-kube-api-access-znh2n\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.130926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.130874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tmp-dir\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.130926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.130894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-home\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.131161 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.130961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-dshm\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.168489 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.168460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:09.231599 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.231561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tls-certs\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.231773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.231610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znh2n\" (UniqueName: \"kubernetes.io/projected/c620025a-5b51-4c8e-ad60-f0e6452a34d4-kube-api-access-znh2n\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.231773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.231644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tmp-dir\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.231773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.231668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-home\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.231773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.231695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-dshm\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.231773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.231759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-model-cache\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.232149 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.232128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tmp-dir\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.232149 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.232142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-model-cache\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.232240 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.232217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-home\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.233958 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.233934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-dshm\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.234312 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.234293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tls-certs\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.240082 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.240029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znh2n\" (UniqueName: \"kubernetes.io/projected/c620025a-5b51-4c8e-ad60-f0e6452a34d4-kube-api-access-znh2n\") pod \"scheduler-custom-template-test-kserve-86c4fb7489-w66nk\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.299336 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.299302    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:09.320812 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.320706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:09.432303 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.432276    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"]
Jul 27 17:57:09.435101 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:57:09.435072    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc620025a_5b51_4c8e_ad60_f0e6452a34d4.slice/crio-ee082a9ac2cdf2a967904ff87e7b86324d21de05f0924ffbc8fd040ab4f52153 WatchSource:0}: Error finding container ee082a9ac2cdf2a967904ff87e7b86324d21de05f0924ffbc8fd040ab4f52153: Status 404 returned error can't find the container with id ee082a9ac2cdf2a967904ff87e7b86324d21de05f0924ffbc8fd040ab4f52153
Jul 27 17:57:09.437482 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.437464    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 17:57:09.828780 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:09.828747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" event={"ID":"c620025a-5b51-4c8e-ad60-f0e6452a34d4","Type":"ContainerStarted","Data":"ee082a9ac2cdf2a967904ff87e7b86324d21de05f0924ffbc8fd040ab4f52153"}
Jul 27 17:57:11.841238 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:11.841202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" event={"ID":"c620025a-5b51-4c8e-ad60-f0e6452a34d4","Type":"ContainerStarted","Data":"adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198"}
Jul 27 17:57:11.859582 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:11.859516    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" podStartSLOduration=2.305902501 podStartE2EDuration="3.859500717s" totalImagesPullingTime="1.553598216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:57:11.859289882 +0000 UTC m=+1699.937722306" watchObservedRunningTime="2026-07-27 17:57:11.859500717 +0000 UTC m=+1699.937933141"
Jul 27 17:57:12.299590 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:12.299558    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:15.107586 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:15.107486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:18.029752 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:18.029713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:19.299883 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:19.299844    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:19.312859 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:19.312831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:19.892772 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:19.892742    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:33.474666 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:33.474634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:42.646077 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.646034    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"]
Jul 27 17:57:42.652826 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.652797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:42.656561 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:57:42.656523    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 17:57:42.656561 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:57:42.656561 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:57:42.656561 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198" cmd=["/bin/sleep","15"]
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.656581    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" message="" err=<
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:57:42.656594    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerName="main" containerID="cri-o://adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198"
Jul 27 17:57:42.656864 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.656642    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerName="main" containerID="cri-o://adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198" gracePeriod=60
Jul 27 17:57:42.714419 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.714388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:57:42.881990 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.881952    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/health\": dial tcp 10.134.0.40:8000: connect: connection refused"
Jul 27 17:57:42.994376 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.994342    2577 generic.go:356] "Generic (PLEG): container finished" podID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerID="adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198" exitCode=0
Jul 27 17:57:42.994571 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:42.994416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" event={"ID":"c620025a-5b51-4c8e-ad60-f0e6452a34d4","Type":"ContainerDied","Data":"adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198"}
Jul 27 17:57:43.018247 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.018224    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:43.145717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.145686    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-dshm\") pod \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") "
Jul 27 17:57:43.145884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.145725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-model-cache\") pod \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") "
Jul 27 17:57:43.145884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.145769    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tls-certs\") pod \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") "
Jul 27 17:57:43.145884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.145824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tmp-dir\") pod \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") "
Jul 27 17:57:43.145884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.145874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znh2n\" (UniqueName: \"kubernetes.io/projected/c620025a-5b51-4c8e-ad60-f0e6452a34d4-kube-api-access-znh2n\") pod \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") "
Jul 27 17:57:43.146043 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.145931    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-home\") pod \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\" (UID: \"c620025a-5b51-4c8e-ad60-f0e6452a34d4\") "
Jul 27 17:57:43.146043 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.146004    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c620025a-5b51-4c8e-ad60-f0e6452a34d4" (UID: "c620025a-5b51-4c8e-ad60-f0e6452a34d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:57:43.146232 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.146184    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c620025a-5b51-4c8e-ad60-f0e6452a34d4" (UID: "c620025a-5b51-4c8e-ad60-f0e6452a34d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:57:43.146301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.146250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-home" (OuterVolumeSpecName: "home") pod "c620025a-5b51-4c8e-ad60-f0e6452a34d4" (UID: "c620025a-5b51-4c8e-ad60-f0e6452a34d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:57:43.146301 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.146270    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:57:43.148015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.147993    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-dshm" (OuterVolumeSpecName: "dshm") pod "c620025a-5b51-4c8e-ad60-f0e6452a34d4" (UID: "c620025a-5b51-4c8e-ad60-f0e6452a34d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:57:43.148091 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.148045    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c620025a-5b51-4c8e-ad60-f0e6452a34d4" (UID: "c620025a-5b51-4c8e-ad60-f0e6452a34d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:57:43.148091 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.148058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c620025a-5b51-4c8e-ad60-f0e6452a34d4-kube-api-access-znh2n" (OuterVolumeSpecName: "kube-api-access-znh2n") pod "c620025a-5b51-4c8e-ad60-f0e6452a34d4" (UID: "c620025a-5b51-4c8e-ad60-f0e6452a34d4"). InnerVolumeSpecName "kube-api-access-znh2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:57:43.247622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.247585    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:57:43.247622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.247617    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:57:43.247622 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.247627    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:57:43.247862 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.247635    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znh2n\" (UniqueName: \"kubernetes.io/projected/c620025a-5b51-4c8e-ad60-f0e6452a34d4-kube-api-access-znh2n\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:57:43.247862 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:43.247644    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c620025a-5b51-4c8e-ad60-f0e6452a34d4-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:57:44.000884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:44.000853    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"
Jul 27 17:57:44.000884 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:44.000865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk" event={"ID":"c620025a-5b51-4c8e-ad60-f0e6452a34d4","Type":"ContainerDied","Data":"ee082a9ac2cdf2a967904ff87e7b86324d21de05f0924ffbc8fd040ab4f52153"}
Jul 27 17:57:44.001399 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:44.000975    2577 scope.go:117] "RemoveContainer" containerID="adc1121fab5e32506abac78dbc632e93f73fb10773879b0f543fff5b137d4198"
Jul 27 17:57:44.021950 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:44.021923    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"]
Jul 27 17:57:44.025897 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:44.025872    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-86c4fb7489-w66nk"]
Jul 27 17:57:44.483317 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:57:44.483274    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" path="/var/lib/kubelet/pods/c620025a-5b51-4c8e-ad60-f0e6452a34d4/volumes"
Jul 27 17:58:02.721343 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:02.721307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:02.814228 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:02.814182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:08.986666 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:08.986635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:09.575604 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:09.575572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:11.625503 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.625467    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"]
Jul 27 17:58:11.625941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.625870    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerName="main"
Jul 27 17:58:11.625941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.625882    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerName="main"
Jul 27 17:58:11.626047 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.625964    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c620025a-5b51-4c8e-ad60-f0e6452a34d4" containerName="main"
Jul 27 17:58:11.629222 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.629200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.632675 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.632646    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-dockercfg-cftqx\""
Jul 27 17:58:11.633635 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.633611    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Jul 27 17:58:11.633918 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.633900    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-qlspx\""
Jul 27 17:58:11.633918 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.633912    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Jul 27 17:58:11.634086 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.633913    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Jul 27 17:58:11.645401 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.645373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"]
Jul 27 17:58:11.677127 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.677092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:11.682398 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.682373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.682547 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.682418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.682547 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.682499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-home\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.682665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.682560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.682665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.682613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng6pc\" (UniqueName: \"kubernetes.io/projected/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-kube-api-access-ng6pc\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.682665 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.682646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.783772 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.783737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.783923 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.783780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng6pc\" (UniqueName: \"kubernetes.io/projected/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-kube-api-access-ng6pc\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.783923 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.783806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.783923 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.783850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.783923 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.783884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.784090 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.783923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-home\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.784282 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.784253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.784350 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.784283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.784350 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.784304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-home\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.786195 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.786179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.786322 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.786276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.792321 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.792299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng6pc\" (UniqueName: \"kubernetes.io/projected/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-kube-api-access-ng6pc\") pod \"scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.847951 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.847919    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"]
Jul 27 17:58:11.856004 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.855978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:11.859551 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.859471    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-v6rnc\""
Jul 27 17:58:11.861162 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.861133    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"]
Jul 27 17:58:11.885207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.885128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1852b28-29c3-4fe8-8510-e96f361b9cd5-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:11.885207 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.885186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mx55\" (UniqueName: \"kubernetes.io/projected/c1852b28-29c3-4fe8-8510-e96f361b9cd5-kube-api-access-5mx55\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:11.901361 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.901330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:11.939931 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.939901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:11.986592 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.986548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mx55\" (UniqueName: \"kubernetes.io/projected/c1852b28-29c3-4fe8-8510-e96f361b9cd5-kube-api-access-5mx55\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:11.986739 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.986661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1852b28-29c3-4fe8-8510-e96f361b9cd5-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:11.989177 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.989142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1852b28-29c3-4fe8-8510-e96f361b9cd5-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:11.995746 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:11.995714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mx55\" (UniqueName: \"kubernetes.io/projected/c1852b28-29c3-4fe8-8510-e96f361b9cd5-kube-api-access-5mx55\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:12.078538 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:12.078484    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"]
Jul 27 17:58:12.080770 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:58:12.080742    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23e61fd_b4e1_4cfa_a06d_66ee44d21d78.slice/crio-1fb328f0b5e2676ccbad33dfcae42bb62048729e31096836fad88c76b7bce25b WatchSource:0}: Error finding container 1fb328f0b5e2676ccbad33dfcae42bb62048729e31096836fad88c76b7bce25b: Status 404 returned error can't find the container with id 1fb328f0b5e2676ccbad33dfcae42bb62048729e31096836fad88c76b7bce25b
Jul 27 17:58:12.130764 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:12.130726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" event={"ID":"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78","Type":"ContainerStarted","Data":"1fb328f0b5e2676ccbad33dfcae42bb62048729e31096836fad88c76b7bce25b"}
Jul 27 17:58:12.170071 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:12.169991    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:12.302035 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:12.302005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"]
Jul 27 17:58:12.303848 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:58:12.303824    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1852b28_29c3_4fe8_8510_e96f361b9cd5.slice/crio-4bf696610f231d9e28886dadab48ce8ec60c8500728bbfd3faa4a06a10dfde07 WatchSource:0}: Error finding container 4bf696610f231d9e28886dadab48ce8ec60c8500728bbfd3faa4a06a10dfde07: Status 404 returned error can't find the container with id 4bf696610f231d9e28886dadab48ce8ec60c8500728bbfd3faa4a06a10dfde07
Jul 27 17:58:13.139092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:13.139054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" event={"ID":"c1852b28-29c3-4fe8-8510-e96f361b9cd5","Type":"ContainerStarted","Data":"0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10"}
Jul 27 17:58:13.139598 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:13.139115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" event={"ID":"c1852b28-29c3-4fe8-8510-e96f361b9cd5","Type":"ContainerStarted","Data":"4bf696610f231d9e28886dadab48ce8ec60c8500728bbfd3faa4a06a10dfde07"}
Jul 27 17:58:13.139598 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:13.139224    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:13.155311 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:13.155265    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" podStartSLOduration=2.155251717 podStartE2EDuration="2.155251717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:58:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:58:13.154829131 +0000 UTC m=+1761.233261555" watchObservedRunningTime="2026-07-27 17:58:13.155251717 +0000 UTC m=+1761.233684182"
Jul 27 17:58:14.144792 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:14.144758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" event={"ID":"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78","Type":"ContainerStarted","Data":"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"}
Jul 27 17:58:14.145370 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:14.144843    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:14.749852 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:14.749818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:14.809414 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:14.809381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:15.119773 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:15.119693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:15.151696 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:15.151667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" event={"ID":"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78","Type":"ContainerStarted","Data":"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"}
Jul 27 17:58:15.171714 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:15.171659    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" podStartSLOduration=2.685913315 podStartE2EDuration="4.171643478s" totalImagesPullingTime="1.485730163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:58:15.169890794 +0000 UTC m=+1763.248323219" watchObservedRunningTime="2026-07-27 17:58:15.171643478 +0000 UTC m=+1763.250075900"
Jul 27 17:58:15.175915 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:15.175888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:15.940248 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:15.940213    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:19.285648 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:19.285617    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:21.940486 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:21.940449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:21.952961 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:21.952938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:22.193962 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:22.193870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:23.200484 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:23.200457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:44.146846 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:44.146771    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:58:45.758184 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:45.758154    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:45.774013 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:45.773982    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"]
Jul 27 17:58:45.777228 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:45.777202    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"]
Jul 27 17:58:45.786319 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:58:45.786280    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 17:58:45.786319 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:58:45.786319 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:58:45.786319 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60" cmd=["/bin/sleep","15"]
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:45.786340    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" message="" err=<
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:58:45.786358    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="main" containerID="cri-o://64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"
Jul 27 17:58:45.786548 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:45.786414    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="main" containerID="cri-o://64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60" gracePeriod=60
Jul 27 17:58:45.829305 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:45.829272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:58:46.001811 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.001782    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="llm-d-routing-sidecar" containerID="cri-o://7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc" gracePeriod=60
Jul 27 17:58:46.265463 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.265395    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:46.308217 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308179    2577 generic.go:356] "Generic (PLEG): container finished" podID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerID="64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60" exitCode=0
Jul 27 17:58:46.308217 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308214    2577 generic.go:356] "Generic (PLEG): container finished" podID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerID="7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc" exitCode=0
Jul 27 17:58:46.308496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308231    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"
Jul 27 17:58:46.308496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" event={"ID":"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78","Type":"ContainerDied","Data":"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"}
Jul 27 17:58:46.308496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308324    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" event={"ID":"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78","Type":"ContainerDied","Data":"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"}
Jul 27 17:58:46.308496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" event={"ID":"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78","Type":"ContainerDied","Data":"1fb328f0b5e2676ccbad33dfcae42bb62048729e31096836fad88c76b7bce25b"}
Jul 27 17:58:46.308496 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.308372    2577 scope.go:117] "RemoveContainer" containerID="64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"
Jul 27 17:58:46.318394 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.318358    2577 scope.go:117] "RemoveContainer" containerID="7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"
Jul 27 17:58:46.325867 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.325846    2577 scope.go:117] "RemoveContainer" containerID="64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"
Jul 27 17:58:46.326093 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:58:46.326075    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60\": container with ID starting with 64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60 not found: ID does not exist" containerID="64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"
Jul 27 17:58:46.326148 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326101    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"} err="failed to get container status \"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60\": rpc error: code = NotFound desc = could not find container \"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60\": container with ID starting with 64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60 not found: ID does not exist"
Jul 27 17:58:46.326148 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326120    2577 scope.go:117] "RemoveContainer" containerID="7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"
Jul 27 17:58:46.326346 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:58:46.326327    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc\": container with ID starting with 7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc not found: ID does not exist" containerID="7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"
Jul 27 17:58:46.326414 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326355    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"} err="failed to get container status \"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc\": rpc error: code = NotFound desc = could not find container \"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc\": container with ID starting with 7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc not found: ID does not exist"
Jul 27 17:58:46.326414 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326376    2577 scope.go:117] "RemoveContainer" containerID="64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"
Jul 27 17:58:46.326659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326641    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60"} err="failed to get container status \"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60\": rpc error: code = NotFound desc = could not find container \"64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60\": container with ID starting with 64df0367800f3043fcab164aa3f9b128314d6ad4b11f202a2a7085e8bb159d60 not found: ID does not exist"
Jul 27 17:58:46.326707 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326660    2577 scope.go:117] "RemoveContainer" containerID="7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"
Jul 27 17:58:46.326918 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.326897    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc"} err="failed to get container status \"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc\": rpc error: code = NotFound desc = could not find container \"7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc\": container with ID starting with 7cb14b2865c90875d7da698e3a0b236a592a302b7849eb985a34f20cb906a3cc not found: ID does not exist"
Jul 27 17:58:46.405945 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.405905    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-model-cache\") pod \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") "
Jul 27 17:58:46.405945 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.405946    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-home\") pod \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") "
Jul 27 17:58:46.406152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.405976    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tls-certs\") pod \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") "
Jul 27 17:58:46.406152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-dshm\") pod \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") "
Jul 27 17:58:46.406152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406033    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng6pc\" (UniqueName: \"kubernetes.io/projected/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-kube-api-access-ng6pc\") pod \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") "
Jul 27 17:58:46.406152 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406138    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tmp-dir\") pod \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\" (UID: \"b23e61fd-b4e1-4cfa-a06d-66ee44d21d78\") "
Jul 27 17:58:46.406368 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406184    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-home" (OuterVolumeSpecName: "home") pod "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" (UID: "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:58:46.406368 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406194    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-model-cache" (OuterVolumeSpecName: "model-cache") pod "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" (UID: "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:58:46.406528 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406487    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" (UID: "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:58:46.406572 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406514    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:58:46.406572 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.406550    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:58:46.408239 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.408219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" (UID: "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:58:46.408339 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.408265    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-kube-api-access-ng6pc" (OuterVolumeSpecName: "kube-api-access-ng6pc") pod "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" (UID: "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78"). InnerVolumeSpecName "kube-api-access-ng6pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:58:46.408339 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.408314    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-dshm" (OuterVolumeSpecName: "dshm") pod "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" (UID: "b23e61fd-b4e1-4cfa-a06d-66ee44d21d78"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:58:46.507000 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.506960    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:58:46.507000 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.506996    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:58:46.507000 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.507005    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:58:46.507000 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.507014    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng6pc\" (UniqueName: \"kubernetes.io/projected/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78-kube-api-access-ng6pc\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:58:46.626582 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.626486    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"]
Jul 27 17:58:46.628066 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:46.628045    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd"]
Jul 27 17:58:47.184466 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:47.184406    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-6c4d7b4898-sqqfd" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 27 17:58:48.483212 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:48.483182    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" path="/var/lib/kubelet/pods/b23e61fd-b4e1-4cfa-a06d-66ee44d21d78/volumes"
Jul 27 17:58:52.441060 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:52.441034    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:58:52.441926 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:52.441902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:58:52.445169 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:52.445142    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 17:58:52.446049 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:52.446029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 17:58:52.446706 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:52.446679    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:58:52.451116 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:52.451100    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 17:58:54.146927 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:54.146825    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" podUID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 17:58:56.050132 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:58:56.050097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:00.794197 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:00.794157    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" podUID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" containerName="main" containerID="cri-o://0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10" gracePeriod=45
Jul 27 17:59:01.051015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.050959    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:59:01.143562 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.143534    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1852b28-29c3-4fe8-8510-e96f361b9cd5-tls-certs\") pod \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") "
Jul 27 17:59:01.143721 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.143579    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mx55\" (UniqueName: \"kubernetes.io/projected/c1852b28-29c3-4fe8-8510-e96f361b9cd5-kube-api-access-5mx55\") pod \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\" (UID: \"c1852b28-29c3-4fe8-8510-e96f361b9cd5\") "
Jul 27 17:59:01.145693 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.145658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1852b28-29c3-4fe8-8510-e96f361b9cd5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c1852b28-29c3-4fe8-8510-e96f361b9cd5" (UID: "c1852b28-29c3-4fe8-8510-e96f361b9cd5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:59:01.145839 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.145771    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1852b28-29c3-4fe8-8510-e96f361b9cd5-kube-api-access-5mx55" (OuterVolumeSpecName: "kube-api-access-5mx55") pod "c1852b28-29c3-4fe8-8510-e96f361b9cd5" (UID: "c1852b28-29c3-4fe8-8510-e96f361b9cd5"). InnerVolumeSpecName "kube-api-access-5mx55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:01.244507 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.244464    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1852b28-29c3-4fe8-8510-e96f361b9cd5-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:01.244507 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.244501    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mx55\" (UniqueName: \"kubernetes.io/projected/c1852b28-29c3-4fe8-8510-e96f361b9cd5-kube-api-access-5mx55\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:01.379230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.379134    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" containerID="0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10" exitCode=0
Jul 27 17:59:01.379230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.379176    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"
Jul 27 17:59:01.379230 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.379222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" event={"ID":"c1852b28-29c3-4fe8-8510-e96f361b9cd5","Type":"ContainerDied","Data":"0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10"}
Jul 27 17:59:01.379494 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.379271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j" event={"ID":"c1852b28-29c3-4fe8-8510-e96f361b9cd5","Type":"ContainerDied","Data":"4bf696610f231d9e28886dadab48ce8ec60c8500728bbfd3faa4a06a10dfde07"}
Jul 27 17:59:01.379494 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.379297    2577 scope.go:117] "RemoveContainer" containerID="0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10"
Jul 27 17:59:01.388155 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.388136    2577 scope.go:117] "RemoveContainer" containerID="0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10"
Jul 27 17:59:01.388393 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:59:01.388376    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10\": container with ID starting with 0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10 not found: ID does not exist" containerID="0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10"
Jul 27 17:59:01.388560 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.388401    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10"} err="failed to get container status \"0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10\": rpc error: code = NotFound desc = could not find container \"0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10\": container with ID starting with 0caab17f7d4e618e43c877cbad7a31189e1056051f428c3742772b3fe5c76e10 not found: ID does not exist"
Jul 27 17:59:01.400941 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.400911    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"]
Jul 27 17:59:01.402667 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:01.402646    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-f9vfv6j"]
Jul 27 17:59:02.483541 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:02.483510    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" path="/var/lib/kubelet/pods/c1852b28-29c3-4fe8-8510-e96f361b9cd5/volumes"
Jul 27 17:59:05.761791 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:05.761755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:05.860081 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:05.860023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:11.805550 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:11.805514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:17.440339 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:17.440304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:17.500983 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:17.500951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:20.618002 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.617960    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"]
Jul 27 17:59:20.618942 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.618917    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="llm-d-routing-sidecar"
Jul 27 17:59:20.618942 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.618941    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="llm-d-routing-sidecar"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.618962    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="main"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.618968    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="main"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.618975    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" containerName="main"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.618982    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" containerName="main"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.619052    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1852b28-29c3-4fe8-8510-e96f361b9cd5" containerName="main"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.619060    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="llm-d-routing-sidecar"
Jul 27 17:59:20.619092 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.619074    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b23e61fd-b4e1-4cfa-a06d-66ee44d21d78" containerName="main"
Jul 27 17:59:20.623840 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.623819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.626709 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.626687    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Jul 27 17:59:20.626827 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.626799    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-zz7t6\""
Jul 27 17:59:20.626827 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.626808    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Jul 27 17:59:20.626948 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.626914    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-dockercfg-xhspj\""
Jul 27 17:59:20.628015 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.627991    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 27 17:59:20.634146 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.634123    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"]
Jul 27 17:59:20.667983 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.667950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:20.722920 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.722873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqlnp\" (UniqueName: \"kubernetes.io/projected/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-kube-api-access-nqlnp\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.723083 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.722934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.723083 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.722963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.723083 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.722986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.723083 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.723004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-home\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.723083 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.723068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824203 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqlnp\" (UniqueName: \"kubernetes.io/projected/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-kube-api-access-nqlnp\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824378 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824594 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-home\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824594 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.824995 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.824879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.825126 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.825063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-home\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.825206 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.825184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.827307 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.827262    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.827420 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.827328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.837719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.837695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqlnp\" (UniqueName: \"kubernetes.io/projected/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-kube-api-access-nqlnp\") pod \"scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:20.842590 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.842566    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"]
Jul 27 17:59:20.846638 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.846617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:20.849183 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.849164    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-2stgs\""
Jul 27 17:59:20.854743 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.854349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"]
Jul 27 17:59:20.864197 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.864165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:20.925056 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.924986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hsp\" (UniqueName: \"kubernetes.io/projected/e2efb919-e768-4fb8-97b6-377873150da2-kube-api-access-n2hsp\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedtt84g\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:20.925056 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.925028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2efb919-e768-4fb8-97b6-377873150da2-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedtt84g\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:20.934938 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:20.934908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:21.026017 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.025980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hsp\" (UniqueName: \"kubernetes.io/projected/e2efb919-e768-4fb8-97b6-377873150da2-kube-api-access-n2hsp\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedtt84g\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:21.026213 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.026031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2efb919-e768-4fb8-97b6-377873150da2-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedtt84g\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:21.028934 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.028906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2efb919-e768-4fb8-97b6-377873150da2-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedtt84g\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:21.035022 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.034959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hsp\" (UniqueName: \"kubernetes.io/projected/e2efb919-e768-4fb8-97b6-377873150da2-kube-api-access-n2hsp\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedtt84g\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:21.076550 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.076523    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"]
Jul 27 17:59:21.079090 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:59:21.079067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd277e2d7_0e32_4a2e_8af7_dde0f4e74810.slice/crio-bc384419bfdcbe55a316dcc248f1e3bd8a7062391fdd1fcbaab08be0c91d97c8 WatchSource:0}: Error finding container bc384419bfdcbe55a316dcc248f1e3bd8a7062391fdd1fcbaab08be0c91d97c8: Status 404 returned error can't find the container with id bc384419bfdcbe55a316dcc248f1e3bd8a7062391fdd1fcbaab08be0c91d97c8
Jul 27 17:59:21.162717 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.162690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:21.294192 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.294167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"]
Jul 27 17:59:21.295799 ip-10-0-135-40 kubenswrapper[2577]: W0727 17:59:21.295778    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2efb919_e768_4fb8_97b6_377873150da2.slice/crio-e6cf867db3275fbf337aff9f8b5c550347439d860e2a64edb6a3315b6f6ddc2e WatchSource:0}: Error finding container e6cf867db3275fbf337aff9f8b5c550347439d860e2a64edb6a3315b6f6ddc2e: Status 404 returned error can't find the container with id e6cf867db3275fbf337aff9f8b5c550347439d860e2a64edb6a3315b6f6ddc2e
Jul 27 17:59:21.472167 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.472129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" event={"ID":"e2efb919-e768-4fb8-97b6-377873150da2","Type":"ContainerStarted","Data":"c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b"}
Jul 27 17:59:21.472370 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.472196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" event={"ID":"e2efb919-e768-4fb8-97b6-377873150da2","Type":"ContainerStarted","Data":"e6cf867db3275fbf337aff9f8b5c550347439d860e2a64edb6a3315b6f6ddc2e"}
Jul 27 17:59:21.472370 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.472226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:21.473625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.473595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" event={"ID":"d277e2d7-0e32-4a2e-8af7-dde0f4e74810","Type":"ContainerStarted","Data":"a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d"}
Jul 27 17:59:21.473902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.473639    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" event={"ID":"d277e2d7-0e32-4a2e-8af7-dde0f4e74810","Type":"ContainerStarted","Data":"bc384419bfdcbe55a316dcc248f1e3bd8a7062391fdd1fcbaab08be0c91d97c8"}
Jul 27 17:59:21.473902 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.473660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:21.487524 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:21.487471    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" podStartSLOduration=1.4874535500000001 podStartE2EDuration="1.48745355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:59:21.48681784 +0000 UTC m=+1829.565250264" watchObservedRunningTime="2026-07-27 17:59:21.48745355 +0000 UTC m=+1829.565885971"
Jul 27 17:59:22.483133 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:22.483100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" event={"ID":"d277e2d7-0e32-4a2e-8af7-dde0f4e74810","Type":"ContainerStarted","Data":"507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"}
Jul 27 17:59:22.526192 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:22.526144    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" podStartSLOduration=2.526127653 podStartE2EDuration="2.526127653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:59:22.52558891 +0000 UTC m=+1830.604021333" watchObservedRunningTime="2026-07-27 17:59:22.526127653 +0000 UTC m=+1830.604560098"
Jul 27 17:59:22.935399 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:22.935305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:28.522580 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:28.522549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:30.935495 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:30.935458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:30.947871 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:30.947843    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:31.536732 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:31.536701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:31.537965 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:31.537945    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:52.484128 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:52.484102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 17:59:54.192918 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.192884    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"]
Jul 27 17:59:54.195942 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.195916    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"]
Jul 27 17:59:54.198157 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.198131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:54.202323 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:59:54.202296    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 17:59:54.202323 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:59:54.202323 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:59:54.202323 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465" cmd=["/bin/sleep","15"]
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.202336    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" message="" err=<
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:59:54.202349    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="main" containerID="cri-o://507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"
Jul 27 17:59:54.202600 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.202382    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="main" containerID="cri-o://507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465" gracePeriod=60
Jul 27 17:59:54.292634 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.292604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 17:59:54.526044 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.526003    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Jul 27 17:59:54.638380 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.638348    2577 generic.go:356] "Generic (PLEG): container finished" podID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerID="507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465" exitCode=0
Jul 27 17:59:54.638567 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.638416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" event={"ID":"d277e2d7-0e32-4a2e-8af7-dde0f4e74810","Type":"ContainerDied","Data":"507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"}
Jul 27 17:59:54.816292 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:54.816262    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="llm-d-routing-sidecar" containerID="cri-o://a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d" gracePeriod=60
Jul 27 17:59:55.073037 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.072976    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:55.127193 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127165    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-home\") pod \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") "
Jul 27 17:59:55.127391 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127264    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-model-cache\") pod \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") "
Jul 27 17:59:55.127391 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127324    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqlnp\" (UniqueName: \"kubernetes.io/projected/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-kube-api-access-nqlnp\") pod \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") "
Jul 27 17:59:55.127391 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-dshm\") pod \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") "
Jul 27 17:59:55.127391 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127386    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tls-certs\") pod \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") "
Jul 27 17:59:55.127659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127457    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tmp-dir\") pod \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\" (UID: \"d277e2d7-0e32-4a2e-8af7-dde0f4e74810\") "
Jul 27 17:59:55.127659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127465    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-home" (OuterVolumeSpecName: "home") pod "d277e2d7-0e32-4a2e-8af7-dde0f4e74810" (UID: "d277e2d7-0e32-4a2e-8af7-dde0f4e74810"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:55.127659 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127550    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-model-cache" (OuterVolumeSpecName: "model-cache") pod "d277e2d7-0e32-4a2e-8af7-dde0f4e74810" (UID: "d277e2d7-0e32-4a2e-8af7-dde0f4e74810"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:55.127813 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127783    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:55.127813 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127801    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:55.127813 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.127803    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d277e2d7-0e32-4a2e-8af7-dde0f4e74810" (UID: "d277e2d7-0e32-4a2e-8af7-dde0f4e74810"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:55.129606 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.129583    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-dshm" (OuterVolumeSpecName: "dshm") pod "d277e2d7-0e32-4a2e-8af7-dde0f4e74810" (UID: "d277e2d7-0e32-4a2e-8af7-dde0f4e74810"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 17:59:55.129719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.129653    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d277e2d7-0e32-4a2e-8af7-dde0f4e74810" (UID: "d277e2d7-0e32-4a2e-8af7-dde0f4e74810"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 17:59:55.129719 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.129664    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-kube-api-access-nqlnp" (OuterVolumeSpecName: "kube-api-access-nqlnp") pod "d277e2d7-0e32-4a2e-8af7-dde0f4e74810" (UID: "d277e2d7-0e32-4a2e-8af7-dde0f4e74810"). InnerVolumeSpecName "kube-api-access-nqlnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 17:59:55.228593 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.228563    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqlnp\" (UniqueName: \"kubernetes.io/projected/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-kube-api-access-nqlnp\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:55.228593 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.228589    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:55.228593 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.228600    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:55.229008 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.228610    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d277e2d7-0e32-4a2e-8af7-dde0f4e74810-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 17:59:55.644795 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.644757    2577 generic.go:356] "Generic (PLEG): container finished" podID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerID="a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d" exitCode=0
Jul 27 17:59:55.644966 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.644819    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"
Jul 27 17:59:55.644966 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.644845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" event={"ID":"d277e2d7-0e32-4a2e-8af7-dde0f4e74810","Type":"ContainerDied","Data":"a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d"}
Jul 27 17:59:55.644966 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.644907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh" event={"ID":"d277e2d7-0e32-4a2e-8af7-dde0f4e74810","Type":"ContainerDied","Data":"bc384419bfdcbe55a316dcc248f1e3bd8a7062391fdd1fcbaab08be0c91d97c8"}
Jul 27 17:59:55.644966 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.644940    2577 scope.go:117] "RemoveContainer" containerID="507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"
Jul 27 17:59:55.654187 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.654172    2577 scope.go:117] "RemoveContainer" containerID="a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d"
Jul 27 17:59:55.661564 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.661547    2577 scope.go:117] "RemoveContainer" containerID="507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"
Jul 27 17:59:55.661847 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:59:55.661829    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465\": container with ID starting with 507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465 not found: ID does not exist" containerID="507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"
Jul 27 17:59:55.661901 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.661857    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465"} err="failed to get container status \"507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465\": rpc error: code = NotFound desc = could not find container \"507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465\": container with ID starting with 507462f9e55a663fe7667c7b6962d5066c94ddb896cb7610c78953aa03508465 not found: ID does not exist"
Jul 27 17:59:55.661901 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.661874    2577 scope.go:117] "RemoveContainer" containerID="a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d"
Jul 27 17:59:55.662092 ip-10-0-135-40 kubenswrapper[2577]: E0727 17:59:55.662075    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d\": container with ID starting with a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d not found: ID does not exist" containerID="a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d"
Jul 27 17:59:55.662134 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.662098    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d"} err="failed to get container status \"a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d\": rpc error: code = NotFound desc = could not find container \"a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d\": container with ID starting with a54dcc1cce84ae1c03ce832271c6046ff55eb3fbbd57baf35cafd4dc45767b4d not found: ID does not exist"
Jul 27 17:59:55.668908 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.668886    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"]
Jul 27 17:59:55.675747 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:55.675725    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-5d7b9b9c45-8zcjh"]
Jul 27 17:59:56.483625 ip-10-0-135-40 kubenswrapper[2577]: I0727 17:59:56.483591    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" path="/var/lib/kubelet/pods/d277e2d7-0e32-4a2e-8af7-dde0f4e74810/volumes"
Jul 27 18:00:02.110848 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:02.110814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:02.484511 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:02.484474    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" podUID="e2efb919-e768-4fb8-97b6-377873150da2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:00:03.001788 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:03.001754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:09.214560 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.214513    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" podUID="e2efb919-e768-4fb8-97b6-377873150da2" containerName="main" containerID="cri-o://c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b" gracePeriod=45
Jul 27 18:00:09.463211 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.463188    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 18:00:09.552962 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.552874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2hsp\" (UniqueName: \"kubernetes.io/projected/e2efb919-e768-4fb8-97b6-377873150da2-kube-api-access-n2hsp\") pod \"e2efb919-e768-4fb8-97b6-377873150da2\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") "
Jul 27 18:00:09.553124 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.552984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2efb919-e768-4fb8-97b6-377873150da2-tls-certs\") pod \"e2efb919-e768-4fb8-97b6-377873150da2\" (UID: \"e2efb919-e768-4fb8-97b6-377873150da2\") "
Jul 27 18:00:09.555033 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.555001    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2efb919-e768-4fb8-97b6-377873150da2-kube-api-access-n2hsp" (OuterVolumeSpecName: "kube-api-access-n2hsp") pod "e2efb919-e768-4fb8-97b6-377873150da2" (UID: "e2efb919-e768-4fb8-97b6-377873150da2"). InnerVolumeSpecName "kube-api-access-n2hsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:00:09.555033 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.555019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2efb919-e768-4fb8-97b6-377873150da2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e2efb919-e768-4fb8-97b6-377873150da2" (UID: "e2efb919-e768-4fb8-97b6-377873150da2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:00:09.654253 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.654222    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2efb919-e768-4fb8-97b6-377873150da2-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:00:09.654253 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.654251    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hsp\" (UniqueName: \"kubernetes.io/projected/e2efb919-e768-4fb8-97b6-377873150da2-kube-api-access-n2hsp\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:00:09.711619 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.711587    2577 generic.go:356] "Generic (PLEG): container finished" podID="e2efb919-e768-4fb8-97b6-377873150da2" containerID="c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b" exitCode=0
Jul 27 18:00:09.711774 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.711637    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"
Jul 27 18:00:09.711774 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.711655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" event={"ID":"e2efb919-e768-4fb8-97b6-377873150da2","Type":"ContainerDied","Data":"c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b"}
Jul 27 18:00:09.711774 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.711697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g" event={"ID":"e2efb919-e768-4fb8-97b6-377873150da2","Type":"ContainerDied","Data":"e6cf867db3275fbf337aff9f8b5c550347439d860e2a64edb6a3315b6f6ddc2e"}
Jul 27 18:00:09.711774 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.711717    2577 scope.go:117] "RemoveContainer" containerID="c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b"
Jul 27 18:00:09.720488 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.720471    2577 scope.go:117] "RemoveContainer" containerID="c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b"
Jul 27 18:00:09.720725 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:00:09.720706    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b\": container with ID starting with c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b not found: ID does not exist" containerID="c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b"
Jul 27 18:00:09.720791 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.720736    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b"} err="failed to get container status \"c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b\": rpc error: code = NotFound desc = could not find container \"c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b\": container with ID starting with c7cba72b34f84e7eba883787c0b7b384fe33d0eb24b85171dc9f19fd24cad82b not found: ID does not exist"
Jul 27 18:00:09.732884 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.732860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"]
Jul 27 18:00:09.738349 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:09.738326    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedtt84g"]
Jul 27 18:00:10.483548 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:10.483514    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2efb919-e768-4fb8-97b6-377873150da2" path="/var/lib/kubelet/pods/e2efb919-e768-4fb8-97b6-377873150da2/volumes"
Jul 27 18:00:13.772014 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:13.771939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:13.870161 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:13.870124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:19.905593 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:19.905530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:25.048628 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.048590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:25.394706 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.394632    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"]
Jul 27 18:00:25.395092 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395075    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="llm-d-routing-sidecar"
Jul 27 18:00:25.395145 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395098    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="llm-d-routing-sidecar"
Jul 27 18:00:25.395145 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395113    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2efb919-e768-4fb8-97b6-377873150da2" containerName="main"
Jul 27 18:00:25.395145 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395122    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2efb919-e768-4fb8-97b6-377873150da2" containerName="main"
Jul 27 18:00:25.395238 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395155    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="main"
Jul 27 18:00:25.395238 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395166    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="main"
Jul 27 18:00:25.395300 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="main"
Jul 27 18:00:25.395300 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395272    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2efb919-e768-4fb8-97b6-377873150da2" containerName="main"
Jul 27 18:00:25.395300 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.395281    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d277e2d7-0e32-4a2e-8af7-dde0f4e74810" containerName="llm-d-routing-sidecar"
Jul 27 18:00:25.398044 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.398023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.403053 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.403028    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-q7zbj\""
Jul 27 18:00:25.403212 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.403051    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Jul 27 18:00:25.403325 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.403084    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Jul 27 18:00:25.403325 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.403090    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Jul 27 18:00:25.403325 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.403095    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-rvs2g\""
Jul 27 18:00:25.411585 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.411565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"]
Jul 27 18:00:25.438629 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.438600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:25.529870 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.529837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:25.594320 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.594285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd7gl\" (UniqueName: \"kubernetes.io/projected/abdeb930-356a-40d4-80a4-ebf15e9c33ed-kube-api-access-nd7gl\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.594501 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.594364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdeb930-356a-40d4-80a4-ebf15e9c33ed-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.695753 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.695662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd7gl\" (UniqueName: \"kubernetes.io/projected/abdeb930-356a-40d4-80a4-ebf15e9c33ed-kube-api-access-nd7gl\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.695753 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.695748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdeb930-356a-40d4-80a4-ebf15e9c33ed-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.698312 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.698292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdeb930-356a-40d4-80a4-ebf15e9c33ed-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.704328 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.704302    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd7gl\" (UniqueName: \"kubernetes.io/projected/abdeb930-356a-40d4-80a4-ebf15e9c33ed-kube-api-access-nd7gl\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.708007 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.707985    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:25.858769 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:25.858730    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"]
Jul 27 18:00:25.862371 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:00:25.862342    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabdeb930_356a_40d4_80a4_ebf15e9c33ed.slice/crio-cf23f0acd847695285e938edff02ca0d9bb39366795b4487da3bc8882d76684e WatchSource:0}: Error finding container cf23f0acd847695285e938edff02ca0d9bb39366795b4487da3bc8882d76684e: Status 404 returned error can't find the container with id cf23f0acd847695285e938edff02ca0d9bb39366795b4487da3bc8882d76684e
Jul 27 18:00:26.344397 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:26.344365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:26.445190 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:26.445154    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:26.797820 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:26.797781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" event={"ID":"abdeb930-356a-40d4-80a4-ebf15e9c33ed","Type":"ContainerStarted","Data":"a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4"}
Jul 27 18:00:26.798015 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:26.797846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" event={"ID":"abdeb930-356a-40d4-80a4-ebf15e9c33ed","Type":"ContainerStarted","Data":"cf23f0acd847695285e938edff02ca0d9bb39366795b4487da3bc8882d76684e"}
Jul 27 18:00:26.798015 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:26.797975    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:00:26.818160 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:26.818107    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" podStartSLOduration=1.818092553 podStartE2EDuration="1.818092553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:00:26.81602743 +0000 UTC m=+1894.894459850" watchObservedRunningTime="2026-07-27 18:00:26.818092553 +0000 UTC m=+1894.896524975"
Jul 27 18:00:36.132139 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:36.132094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:00:57.805233 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:00:57.805202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:01:24.474492 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:01:24.474458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:02:20.300769 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:20.300734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:02:25.139348 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:25.139303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:02:25.143542 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:25.143511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"]
Jul 27 18:02:25.215351 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:25.215322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:02:27.805340 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:27.805301    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:02:37.804617 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:37.804578    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:02:40.163951 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.163910    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerName="main" containerID="cri-o://a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4" gracePeriod=45
Jul 27 18:02:40.407159 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.407131    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:02:40.438109 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.438027    2577 generic.go:356] "Generic (PLEG): container finished" podID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerID="a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4" exitCode=0
Jul 27 18:02:40.438109 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.438073    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"
Jul 27 18:02:40.438272 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.438111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" event={"ID":"abdeb930-356a-40d4-80a4-ebf15e9c33ed","Type":"ContainerDied","Data":"a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4"}
Jul 27 18:02:40.438272 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.438162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f" event={"ID":"abdeb930-356a-40d4-80a4-ebf15e9c33ed","Type":"ContainerDied","Data":"cf23f0acd847695285e938edff02ca0d9bb39366795b4487da3bc8882d76684e"}
Jul 27 18:02:40.438272 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.438183    2577 scope.go:117] "RemoveContainer" containerID="a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4"
Jul 27 18:02:40.448000 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.447982    2577 scope.go:117] "RemoveContainer" containerID="a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4"
Jul 27 18:02:40.448285 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:02:40.448266    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4\": container with ID starting with a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4 not found: ID does not exist" containerID="a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4"
Jul 27 18:02:40.448329 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.448296    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4"} err="failed to get container status \"a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4\": rpc error: code = NotFound desc = could not find container \"a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4\": container with ID starting with a3efd2f6aa0db168e4118ee8ba1cf6787b4c43817a16759022fb9625f1bb8ff4 not found: ID does not exist"
Jul 27 18:02:40.471682 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.471654    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdeb930-356a-40d4-80a4-ebf15e9c33ed-tls-certs\") pod \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") "
Jul 27 18:02:40.471829 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.471779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd7gl\" (UniqueName: \"kubernetes.io/projected/abdeb930-356a-40d4-80a4-ebf15e9c33ed-kube-api-access-nd7gl\") pod \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\" (UID: \"abdeb930-356a-40d4-80a4-ebf15e9c33ed\") "
Jul 27 18:02:40.473901 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.473876    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abdeb930-356a-40d4-80a4-ebf15e9c33ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abdeb930-356a-40d4-80a4-ebf15e9c33ed" (UID: "abdeb930-356a-40d4-80a4-ebf15e9c33ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:02:40.473995 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.473972    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdeb930-356a-40d4-80a4-ebf15e9c33ed-kube-api-access-nd7gl" (OuterVolumeSpecName: "kube-api-access-nd7gl") pod "abdeb930-356a-40d4-80a4-ebf15e9c33ed" (UID: "abdeb930-356a-40d4-80a4-ebf15e9c33ed"). InnerVolumeSpecName "kube-api-access-nd7gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:02:40.573099 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.573060    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd7gl\" (UniqueName: \"kubernetes.io/projected/abdeb930-356a-40d4-80a4-ebf15e9c33ed-kube-api-access-nd7gl\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:02:40.573099 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.573091    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abdeb930-356a-40d4-80a4-ebf15e9c33ed-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:02:40.753885 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.753851    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"]
Jul 27 18:02:40.757986 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:40.757961    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-866ccd65k9s8f"]
Jul 27 18:02:42.484574 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:02:42.484539    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" path="/var/lib/kubelet/pods/abdeb930-356a-40d4-80a4-ebf15e9c33ed/volumes"
Jul 27 18:03:45.358500 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:45.358401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:03:45.426002 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:45.425967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:03:52.467773 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:52.467734    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:03:52.468633 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:52.468615    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:03:52.473542 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:52.473522    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:03:52.474079 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:52.474049    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:03:52.474981 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:52.474965    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:03:52.480329 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:03:52.480307    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:06:26.675872 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:06:26.675783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:31.665483 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:06:31.665450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:06:39.843651 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:06:39.843613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:08:52.494374 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:08:52.494347    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:08:52.495370 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:08:52.495346    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:08:52.500237 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:08:52.500213    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:08:52.503205 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:08:52.503187    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:08:52.504049 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:08:52.504030    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:08:52.508612 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:08:52.508595    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:11:29.776954 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:11:29.776922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:29.856547 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:11:29.856513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:49.816744 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:11:49.816699    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:11:52.862660 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:11:52.862621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:00.831045 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:00.831011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:01.288531 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.288492    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"]
Jul 27 18:12:01.289121 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.289081    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerName="main"
Jul 27 18:12:01.289306 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.289281    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerName="main"
Jul 27 18:12:01.289421 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.289404    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdeb930-356a-40d4-80a4-ebf15e9c33ed" containerName="main"
Jul 27 18:12:01.292646 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.292625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.295517 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.295487    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"kube-root-ca.crt\""
Jul 27 18:12:01.295645 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.295488    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"autoscale-keda-deploy-kserve-self-signed-certs\""
Jul 27 18:12:01.296637 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.296614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"autoscale-keda-deploy-epp-sa-dockercfg-x6z98\""
Jul 27 18:12:01.296756 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.296658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"default-dockercfg-4jwxt\""
Jul 27 18:12:01.296756 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.296685    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"openshift-service-ca.crt\""
Jul 27 18:12:01.303541 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.303515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"]
Jul 27 18:12:01.404944 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.404904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkgth\" (UniqueName: \"kubernetes.io/projected/afcfc717-31aa-4e76-9343-938d9e60816c-kube-api-access-dkgth\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.405142 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.404959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcfc717-31aa-4e76-9343-938d9e60816c-tls-certs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.506420 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.506382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkgth\" (UniqueName: \"kubernetes.io/projected/afcfc717-31aa-4e76-9343-938d9e60816c-kube-api-access-dkgth\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.506624 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.506457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcfc717-31aa-4e76-9343-938d9e60816c-tls-certs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.509015 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.508983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcfc717-31aa-4e76-9343-938d9e60816c-tls-certs\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.515932 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.515908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkgth\" (UniqueName: \"kubernetes.io/projected/afcfc717-31aa-4e76-9343-938d9e60816c-kube-api-access-dkgth\") pod \"autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.596466 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.596368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:01.605303 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.605284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:01.741531 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.741492    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"]
Jul 27 18:12:01.744051 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:12:01.744022    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcfc717_31aa_4e76_9343_938d9e60816c.slice/crio-01920337dedab15f6cd368a9823e97b2663a9f1e3d479593dba42ed49c6b1949 WatchSource:0}: Error finding container 01920337dedab15f6cd368a9823e97b2663a9f1e3d479593dba42ed49c6b1949: Status 404 returned error can't find the container with id 01920337dedab15f6cd368a9823e97b2663a9f1e3d479593dba42ed49c6b1949
Jul 27 18:12:01.989850 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.989809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" event={"ID":"afcfc717-31aa-4e76-9343-938d9e60816c","Type":"ContainerStarted","Data":"6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd"}
Jul 27 18:12:01.990362 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.989872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" event={"ID":"afcfc717-31aa-4e76-9343-938d9e60816c","Type":"ContainerStarted","Data":"01920337dedab15f6cd368a9823e97b2663a9f1e3d479593dba42ed49c6b1949"}
Jul 27 18:12:01.990362 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:01.989898    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:12:02.007634 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:02.007556    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" podStartSLOduration=1.007538574 podStartE2EDuration="1.007538574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:12:02.005313238 +0000 UTC m=+2590.083745663" watchObservedRunningTime="2026-07-27 18:12:02.007538574 +0000 UTC m=+2590.085971004"
Jul 27 18:12:11.919548 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:11.919510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:12:32.996331 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:12:32.996242    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:13:03.915490 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:03.915452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:13:52.521335 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:52.521304    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:13:52.522296 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:52.522278    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:13:52.526830 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:52.526808    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:13:52.530578 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:52.530560    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:13:52.531319 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:52.531302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:13:52.535858 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:13:52.535843    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:14:45.845178 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:14:45.845118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:16:26.904717 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:16:26.904684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:33.055812 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:33.055711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:34.850163 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:34.850131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:40.321909 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:40.321845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:41.652290 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:41.652258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:41.970079 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:41.970048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:42.068327 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:42.068288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:45.892200 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:45.892157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:45.994294 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:45.994263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:18:52.546887 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:52.546860    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:18:52.547959 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:52.547930    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:18:52.552969 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:52.552949    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:18:52.557849 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:52.557830    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:18:52.558680 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:52.558662    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:18:52.562973 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:18:52.562953    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:19:01.040523 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:01.040484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:27.482463 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:27.482404    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:27.561228 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:27.561193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:47.547389 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:47.547354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:47.647330 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:47.647297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:54.382559 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:54.382527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:54.606631 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:54.606594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:54.952041 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:54.952005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:55.608060 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:55.608027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:19:59.875180 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:19:59.875125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:01.086251 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:01.086221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:01.186371 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:01.186336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:27.911891 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:27.911851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:28.009488 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:28.009453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:32.318744 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:32.318705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:46.896059 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:46.896021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:46.990643 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:46.990614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:52.918379 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:52.918343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:53.019279 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:53.019245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:53.389420 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:53.389385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:58.992060 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:58.992023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:20:59.143337 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:20:59.143304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:21:17.387232 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:17.387155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:21:31.087656 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.087620    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"]
Jul 27 18:21:31.091636 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.091609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.094324 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.094300    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Jul 27 18:21:31.095158 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.095141    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Jul 27 18:21:31.095268 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.095156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-n86fh\""
Jul 27 18:21:31.095380 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.095367    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Jul 27 18:21:31.101456 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.099282    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"]
Jul 27 18:21:31.118870 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.118829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:21:31.244779 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.244741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.244779 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.244778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxm6g\" (UniqueName: \"kubernetes.io/projected/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-kube-api-access-vxm6g\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.244981 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.244876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.244981 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.244908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.245054 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.244992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.245054 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.245023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.318318 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.318280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:21:31.346130 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346130 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346130 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346130 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxm6g\" (UniqueName: \"kubernetes.io/projected/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-kube-api-access-vxm6g\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346491 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346491 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346586 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346586 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.346658 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.346612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.348367 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.348346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.348671 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.348653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.354723 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.354701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxm6g\" (UniqueName: \"kubernetes.io/projected/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-kube-api-access-vxm6g\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.408403 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.408370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:31.540186 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.540151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"]
Jul 27 18:21:31.543699 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:21:31.543669    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cd6336_24bd_4e9b_8fe3_3070e47efa4e.slice/crio-4f8f2af2f4b57582675815dceea60b6aa9b89f9c27c55198ad491c6a5cbc5c27 WatchSource:0}: Error finding container 4f8f2af2f4b57582675815dceea60b6aa9b89f9c27c55198ad491c6a5cbc5c27: Status 404 returned error can't find the container with id 4f8f2af2f4b57582675815dceea60b6aa9b89f9c27c55198ad491c6a5cbc5c27
Jul 27 18:21:31.611969 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.611938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" event={"ID":"23cd6336-24bd-4e9b-8fe3-3070e47efa4e","Type":"ContainerStarted","Data":"1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be"}
Jul 27 18:21:31.612083 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.612023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" event={"ID":"23cd6336-24bd-4e9b-8fe3-3070e47efa4e","Type":"ContainerStarted","Data":"4f8f2af2f4b57582675815dceea60b6aa9b89f9c27c55198ad491c6a5cbc5c27"}
Jul 27 18:21:31.629045 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:31.628996    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" podStartSLOduration=0.628980011 podStartE2EDuration="628.980011ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:21:31.62725532 +0000 UTC m=+3159.705687744" watchObservedRunningTime="2026-07-27 18:21:31.628980011 +0000 UTC m=+3159.707412434"
Jul 27 18:21:32.409490 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:32.409461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:41.409172 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:41.409128    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:41.422873 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:41.422849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:41.669965 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:41.669890    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:21:45.493924 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:21:45.493891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:04.513770 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.513736    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"]
Jul 27 18:22:04.523282 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:22:04.523249    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 18:22:04.523282 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:22:04.523282 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:22:04.523282 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be" cmd=["/bin/sleep","15"]
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.523300    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" message="" err=<
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:22:04.523316    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerName="main" containerID="cri-o://1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be"
Jul 27 18:22:04.523628 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.523371    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerName="main" containerID="cri-o://1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be" gracePeriod=60
Jul 27 18:22:04.524202 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.523732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:04.624484 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.624452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:04.659537 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.659506    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Jul 27 18:22:04.767748 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.767660    2577 generic.go:356] "Generic (PLEG): container finished" podID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerID="1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be" exitCode=0
Jul 27 18:22:04.767892 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:04.767739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" event={"ID":"23cd6336-24bd-4e9b-8fe3-3070e47efa4e","Type":"ContainerDied","Data":"1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be"}
Jul 27 18:22:05.271365 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.271339    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:22:05.343544 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tls-certs\") pod \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") "
Jul 27 18:22:05.343721 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-model-cache\") pod \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") "
Jul 27 18:22:05.343721 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343619    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-home\") pod \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") "
Jul 27 18:22:05.343721 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343656    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tmp-dir\") pod \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") "
Jul 27 18:22:05.343721 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxm6g\" (UniqueName: \"kubernetes.io/projected/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-kube-api-access-vxm6g\") pod \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") "
Jul 27 18:22:05.343989 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-dshm\") pod \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\" (UID: \"23cd6336-24bd-4e9b-8fe3-3070e47efa4e\") "
Jul 27 18:22:05.343989 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343862    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "23cd6336-24bd-4e9b-8fe3-3070e47efa4e" (UID: "23cd6336-24bd-4e9b-8fe3-3070e47efa4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:22:05.344058 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343979    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-home" (OuterVolumeSpecName: "home") pod "23cd6336-24bd-4e9b-8fe3-3070e47efa4e" (UID: "23cd6336-24bd-4e9b-8fe3-3070e47efa4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:22:05.344058 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.343992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23cd6336-24bd-4e9b-8fe3-3070e47efa4e" (UID: "23cd6336-24bd-4e9b-8fe3-3070e47efa4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:22:05.344143 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.344100    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:22:05.344143 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.344115    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:22:05.345890 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.345863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-dshm" (OuterVolumeSpecName: "dshm") pod "23cd6336-24bd-4e9b-8fe3-3070e47efa4e" (UID: "23cd6336-24bd-4e9b-8fe3-3070e47efa4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:22:05.345890 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.345876    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-kube-api-access-vxm6g" (OuterVolumeSpecName: "kube-api-access-vxm6g") pod "23cd6336-24bd-4e9b-8fe3-3070e47efa4e" (UID: "23cd6336-24bd-4e9b-8fe3-3070e47efa4e"). InnerVolumeSpecName "kube-api-access-vxm6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:22:05.345890 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.345882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23cd6336-24bd-4e9b-8fe3-3070e47efa4e" (UID: "23cd6336-24bd-4e9b-8fe3-3070e47efa4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:22:05.444646 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.444562    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:22:05.444646 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.444590    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxm6g\" (UniqueName: \"kubernetes.io/projected/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-kube-api-access-vxm6g\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:22:05.444646 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.444604    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:22:05.444646 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.444616    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23cd6336-24bd-4e9b-8fe3-3070e47efa4e-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:22:05.773907 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.773879    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"
Jul 27 18:22:05.774311 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.773882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p" event={"ID":"23cd6336-24bd-4e9b-8fe3-3070e47efa4e","Type":"ContainerDied","Data":"4f8f2af2f4b57582675815dceea60b6aa9b89f9c27c55198ad491c6a5cbc5c27"}
Jul 27 18:22:05.774311 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.774001    2577 scope.go:117] "RemoveContainer" containerID="1b43379ddf58d3bdf1b3b5c027dbc905d324c761002c5d17ee79456b374fd0be"
Jul 27 18:22:05.797112 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.797085    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"]
Jul 27 18:22:05.800761 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:05.800737    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5cf996c995ts48p"]
Jul 27 18:22:06.483298 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:06.483260    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" path="/var/lib/kubelet/pods/23cd6336-24bd-4e9b-8fe3-3070e47efa4e/volumes"
Jul 27 18:22:26.281601 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:26.281560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:26.383347 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:26.383310    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:44.532284 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:44.532199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:44.626166 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:44.626129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:51.258949 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:51.258914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:51.999872 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:51.999839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:52.209280 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:52.209245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:55.325935 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:55.325902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:55.383838 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:55.383805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:57.429547 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:57.429511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:22:57.529887 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:22:57.529854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:23:05.389549 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:05.389513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:23:29.797929 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.797894    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"]
Jul 27 18:23:29.798398 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.798285    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerName="main"
Jul 27 18:23:29.798398 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.798297    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerName="main"
Jul 27 18:23:29.798398 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.798368    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cd6336-24bd-4e9b-8fe3-3070e47efa4e" containerName="main"
Jul 27 18:23:29.801500 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.801484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.811502 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.811480    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Jul 27 18:23:29.812587 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.812570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-r7d8l\""
Jul 27 18:23:29.812675 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.812626    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Jul 27 18:23:29.812675 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.812648    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Jul 27 18:23:29.822312 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.822291    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"]
Jul 27 18:23:29.880791 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.880755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.880958 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.880827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.880958 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.880866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.880958 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.880885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmdmb\" (UniqueName: \"kubernetes.io/projected/4f01e1f7-dd8c-4272-b669-e3d7c9302184-kube-api-access-fmdmb\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.880958 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.880922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.881148 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.881000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.938488 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.938454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:23:29.981665 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.981629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.981831 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.981702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.981831 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.981749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.981831 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.981782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.981831 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.981813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.982007 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.981838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmdmb\" (UniqueName: \"kubernetes.io/projected/4f01e1f7-dd8c-4272-b669-e3d7c9302184-kube-api-access-fmdmb\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.982144 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.982122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.982228 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.982201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.982288 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.982231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.984193 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.984172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.984390 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.984372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:29.989254 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:29.989231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmdmb\" (UniqueName: \"kubernetes.io/projected/4f01e1f7-dd8c-4272-b669-e3d7c9302184-kube-api-access-fmdmb\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:30.112016 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:30.111900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:30.140722 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:30.140694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:23:30.252209 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:30.252157    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"]
Jul 27 18:23:30.254531 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:23:30.254504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f01e1f7_dd8c_4272_b669_e3d7c9302184.slice/crio-b812e3359d00effeba9975c9c71a9e3b556f2eef1eb48c199d26bbcbec6c6d06 WatchSource:0}: Error finding container b812e3359d00effeba9975c9c71a9e3b556f2eef1eb48c199d26bbcbec6c6d06: Status 404 returned error can't find the container with id b812e3359d00effeba9975c9c71a9e3b556f2eef1eb48c199d26bbcbec6c6d06
Jul 27 18:23:31.167458 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:31.167418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" event={"ID":"4f01e1f7-dd8c-4272-b669-e3d7c9302184","Type":"ContainerStarted","Data":"e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"}
Jul 27 18:23:31.167851 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:31.167476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" event={"ID":"4f01e1f7-dd8c-4272-b669-e3d7c9302184","Type":"ContainerStarted","Data":"b812e3359d00effeba9975c9c71a9e3b556f2eef1eb48c199d26bbcbec6c6d06"}
Jul 27 18:23:31.185607 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:31.185558    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" podStartSLOduration=2.185538455 podStartE2EDuration="2.185538455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:23:31.184909284 +0000 UTC m=+3279.263341707" watchObservedRunningTime="2026-07-27 18:23:31.185538455 +0000 UTC m=+3279.263970880"
Jul 27 18:23:32.112039 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:32.111999    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:38.081538 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:38.081502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:23:40.113028 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:40.112993    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:40.126077 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:40.126047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:40.220099 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:40.220061    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:23:52.574622 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:52.574588    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:23:52.575656 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:52.575632    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:23:52.580666 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:52.580645    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:23:52.585922 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:52.585893    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:23:52.586851 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:52.586834    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:23:52.592107 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:23:52.592088    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:24:03.620954 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:03.620910    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"]
Jul 27 18:24:03.631546 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:24:03.631494    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 18:24:03.631546 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:24:03.631546 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:24:03.631546 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68" cmd=["/bin/sleep","15"]
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:03.631571    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" message="" err=<
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:24:03.631589    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" podUID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" containerName="main" containerID="cri-o://e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"
Jul 27 18:24:03.631992 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:03.631635    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" podUID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" containerName="main" containerID="cri-o://e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68" gracePeriod=60
Jul 27 18:24:03.633458 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:03.633410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:03.693887 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:03.693854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:03.991882 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:03.991858    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:24:04.092912 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.092873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmdmb\" (UniqueName: \"kubernetes.io/projected/4f01e1f7-dd8c-4272-b669-e3d7c9302184-kube-api-access-fmdmb\") pod \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") "
Jul 27 18:24:04.093081 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.092928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-model-cache\") pod \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") "
Jul 27 18:24:04.093081 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.092980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tls-certs\") pod \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") "
Jul 27 18:24:04.093081 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tmp-dir\") pod \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") "
Jul 27 18:24:04.093235 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-home\") pod \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") "
Jul 27 18:24:04.093235 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093122    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-dshm\") pod \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\" (UID: \"4f01e1f7-dd8c-4272-b669-e3d7c9302184\") "
Jul 27 18:24:04.093340 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093236    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f01e1f7-dd8c-4272-b669-e3d7c9302184" (UID: "4f01e1f7-dd8c-4272-b669-e3d7c9302184"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:24:04.093340 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093305    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f01e1f7-dd8c-4272-b669-e3d7c9302184" (UID: "4f01e1f7-dd8c-4272-b669-e3d7c9302184"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:24:04.093564 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093387    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-home" (OuterVolumeSpecName: "home") pod "4f01e1f7-dd8c-4272-b669-e3d7c9302184" (UID: "4f01e1f7-dd8c-4272-b669-e3d7c9302184"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:24:04.093564 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093521    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:24:04.093564 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093542    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:24:04.093564 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.093556    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:24:04.095248 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.095217    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-dshm" (OuterVolumeSpecName: "dshm") pod "4f01e1f7-dd8c-4272-b669-e3d7c9302184" (UID: "4f01e1f7-dd8c-4272-b669-e3d7c9302184"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:24:04.095345 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.095280    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f01e1f7-dd8c-4272-b669-e3d7c9302184" (UID: "4f01e1f7-dd8c-4272-b669-e3d7c9302184"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:24:04.095345 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.095289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f01e1f7-dd8c-4272-b669-e3d7c9302184-kube-api-access-fmdmb" (OuterVolumeSpecName: "kube-api-access-fmdmb") pod "4f01e1f7-dd8c-4272-b669-e3d7c9302184" (UID: "4f01e1f7-dd8c-4272-b669-e3d7c9302184"). InnerVolumeSpecName "kube-api-access-fmdmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:24:04.194095 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.194005    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f01e1f7-dd8c-4272-b669-e3d7c9302184-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:24:04.194095 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.194038    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmdmb\" (UniqueName: \"kubernetes.io/projected/4f01e1f7-dd8c-4272-b669-e3d7c9302184-kube-api-access-fmdmb\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:24:04.194095 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.194049    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f01e1f7-dd8c-4272-b669-e3d7c9302184-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:24:04.321744 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.321708    2577 generic.go:356] "Generic (PLEG): container finished" podID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" containerID="e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68" exitCode=0
Jul 27 18:24:04.321744 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.321747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" event={"ID":"4f01e1f7-dd8c-4272-b669-e3d7c9302184","Type":"ContainerDied","Data":"e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"}
Jul 27 18:24:04.321964 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.321787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5" event={"ID":"4f01e1f7-dd8c-4272-b669-e3d7c9302184","Type":"ContainerDied","Data":"b812e3359d00effeba9975c9c71a9e3b556f2eef1eb48c199d26bbcbec6c6d06"}
Jul 27 18:24:04.321964 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.321809    2577 scope.go:117] "RemoveContainer" containerID="e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"
Jul 27 18:24:04.321964 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.321752    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"
Jul 27 18:24:04.331896 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.331875    2577 scope.go:117] "RemoveContainer" containerID="e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"
Jul 27 18:24:04.332198 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:24:04.332180    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68\": container with ID starting with e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68 not found: ID does not exist" containerID="e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"
Jul 27 18:24:04.332265 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.332208    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68"} err="failed to get container status \"e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68\": rpc error: code = NotFound desc = could not find container \"e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68\": container with ID starting with e28374e9c9ced4a5f9cd44de34031f1bed8ea24daee1f8c50e904fbd3cc58c68 not found: ID does not exist"
Jul 27 18:24:04.345561 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.345531    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"]
Jul 27 18:24:04.348727 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.348706    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-p4xp5"]
Jul 27 18:24:04.483624 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:04.483592    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" path="/var/lib/kubelet/pods/4f01e1f7-dd8c-4272-b669-e3d7c9302184/volumes"
Jul 27 18:24:24.007882 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:24.007805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:24.108752 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:24.108704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:43.870762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:43.870727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:43.963466 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:43.963415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:47.939643 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:47.939609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:48.191697 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:48.191613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:51.081550 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:51.081515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:54.436261 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:54.436230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:57.032210 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:57.032174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:57.099739 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:57.099708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:24:59.119791 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:24:59.119760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:25:04.979728 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:25:04.979695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:25:13.918047 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:25:13.918011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:25:14.019519 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:25:14.019475    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:26:43.320977 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:26:43.320937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:26:43.381648 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:26:43.381612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:26:50.850085 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:26:50.850052    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"]
Jul 27 18:26:51.386189 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:26:51.386156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:26:51.487176 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:26:51.487146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:26:52.996871 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:26:52.996835    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:27:02.996529 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:02.996489    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:27:03.298608 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:03.298522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:03.395845 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:03.395814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:05.869931 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:05.869865    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" containerName="main" containerID="cri-o://6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd" gracePeriod=45
Jul 27 18:27:06.124133 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.124074    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:27:06.165748 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.165712    2577 generic.go:356] "Generic (PLEG): container finished" podID="afcfc717-31aa-4e76-9343-938d9e60816c" containerID="6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd" exitCode=0
Jul 27 18:27:06.165918 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.165755    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"
Jul 27 18:27:06.165918 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.165795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" event={"ID":"afcfc717-31aa-4e76-9343-938d9e60816c","Type":"ContainerDied","Data":"6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd"}
Jul 27 18:27:06.165918 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.165863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght" event={"ID":"afcfc717-31aa-4e76-9343-938d9e60816c","Type":"ContainerDied","Data":"01920337dedab15f6cd368a9823e97b2663a9f1e3d479593dba42ed49c6b1949"}
Jul 27 18:27:06.165918 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.165901    2577 scope.go:117] "RemoveContainer" containerID="6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd"
Jul 27 18:27:06.174335 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.174317    2577 scope.go:117] "RemoveContainer" containerID="6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd"
Jul 27 18:27:06.174578 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:27:06.174559    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd\": container with ID starting with 6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd not found: ID does not exist" containerID="6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd"
Jul 27 18:27:06.174637 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.174588    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd"} err="failed to get container status \"6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd\": rpc error: code = NotFound desc = could not find container \"6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd\": container with ID starting with 6ab281cd7cacd9e99aed8c1b842d24b6c1744780beceb02747257c309f6ca6dd not found: ID does not exist"
Jul 27 18:27:06.182949 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.182926    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkgth\" (UniqueName: \"kubernetes.io/projected/afcfc717-31aa-4e76-9343-938d9e60816c-kube-api-access-dkgth\") pod \"afcfc717-31aa-4e76-9343-938d9e60816c\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") "
Jul 27 18:27:06.183057 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.182969    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcfc717-31aa-4e76-9343-938d9e60816c-tls-certs\") pod \"afcfc717-31aa-4e76-9343-938d9e60816c\" (UID: \"afcfc717-31aa-4e76-9343-938d9e60816c\") "
Jul 27 18:27:06.185155 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.185131    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afcfc717-31aa-4e76-9343-938d9e60816c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "afcfc717-31aa-4e76-9343-938d9e60816c" (UID: "afcfc717-31aa-4e76-9343-938d9e60816c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:27:06.185155 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.185147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcfc717-31aa-4e76-9343-938d9e60816c-kube-api-access-dkgth" (OuterVolumeSpecName: "kube-api-access-dkgth") pod "afcfc717-31aa-4e76-9343-938d9e60816c" (UID: "afcfc717-31aa-4e76-9343-938d9e60816c"). InnerVolumeSpecName "kube-api-access-dkgth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:27:06.284112 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.284075    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkgth\" (UniqueName: \"kubernetes.io/projected/afcfc717-31aa-4e76-9343-938d9e60816c-kube-api-access-dkgth\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:27:06.284112 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.284104    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afcfc717-31aa-4e76-9343-938d9e60816c-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:27:06.487762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.487729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"]
Jul 27 18:27:06.492130 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:06.492095    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-router-scheduler-79d6bcd899-ndght"]
Jul 27 18:27:08.484127 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:08.484094    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" path="/var/lib/kubelet/pods/afcfc717-31aa-4e76-9343-938d9e60816c/volumes"
Jul 27 18:27:09.460591 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:09.460557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:10.824209 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:10.824143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:10.924262 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:10.924233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:13.415348 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:13.415315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:13.678909 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:13.678793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:15.520996 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:15.520959    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:15.618297 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:15.618266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:21.207932 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.207896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:21.305998 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.305966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:21.311075 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311049    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 27 18:27:21.311423 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311411    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" containerName="main"
Jul 27 18:27:21.311480 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311438    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" containerName="main"
Jul 27 18:27:21.311480 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311461    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" containerName="main"
Jul 27 18:27:21.311480 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311467    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" containerName="main"
Jul 27 18:27:21.311578 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311525    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcfc717-31aa-4e76-9343-938d9e60816c" containerName="main"
Jul 27 18:27:21.311578 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.311534    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f01e1f7-dd8c-4272-b669-e3d7c9302184" containerName="main"
Jul 27 18:27:21.315999 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.315982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.318957 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.318937    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"openshift-service-ca.crt\""
Jul 27 18:27:21.320094 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.320062    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-kserve-self-signed-certs\""
Jul 27 18:27:21.320094 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.320080    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"default-dockercfg-wg99b\""
Jul 27 18:27:21.320274 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.320061    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"kube-root-ca.crt\""
Jul 27 18:27:21.320274 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.320086    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-kserve-mn-dockercfg-8th6f\""
Jul 27 18:27:21.325330 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.325311    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 27 18:27:21.382188 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.382155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:21.423063 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.423025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.423063 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.423061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.423295 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.423098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.423295 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.423123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jdjw\" (UniqueName: \"kubernetes.io/projected/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-kube-api-access-7jdjw\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.423295 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.423198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.423295 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.423213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.435961 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.435919    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"]
Jul 27 18:27:21.444236 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.444205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.446958 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.446934    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-epp-sa-dockercfg-sbj9d\""
Jul 27 18:27:21.452452 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.452386    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"]
Jul 27 18:27:21.473064 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.473013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:21.524627 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.524627 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-tls-certs\") pod \"autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.524840 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdjw\" (UniqueName: \"kubernetes.io/projected/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-kube-api-access-7jdjw\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.524840 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5hvf\" (UniqueName: \"kubernetes.io/projected/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-kube-api-access-n5hvf\") pod \"autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.524921 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.524921 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.524995 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.524995 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.524976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.525231 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.525209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.525293 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.525244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.525342 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.525308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.527185 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.527167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.527301 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.527280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.532972 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.532944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdjw\" (UniqueName: \"kubernetes.io/projected/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-kube-api-access-7jdjw\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.626381 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.626336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-tls-certs\") pod \"autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.626381 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.626374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5hvf\" (UniqueName: \"kubernetes.io/projected/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-kube-api-access-n5hvf\") pod \"autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.626656 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.626545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:21.628944 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.628920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-tls-certs\") pod \"autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.634920 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.634893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5hvf\" (UniqueName: \"kubernetes.io/projected/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-kube-api-access-n5hvf\") pod \"autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.757490 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.757385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:21.767777 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.767751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 27 18:27:21.770439 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:27:21.770404    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b73dd00_c4d0_45d1_9de4_4d7e510c544d.slice/crio-f7932c2de71ff3cc346ce9ff742f0474ba5ea857a142c9fb9793d5753dcdc408 WatchSource:0}: Error finding container f7932c2de71ff3cc346ce9ff742f0474ba5ea857a142c9fb9793d5753dcdc408: Status 404 returned error can't find the container with id f7932c2de71ff3cc346ce9ff742f0474ba5ea857a142c9fb9793d5753dcdc408
Jul 27 18:27:21.901139 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:21.901116    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"]
Jul 27 18:27:21.903117 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:27:21.903089    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aaa93b7_c8e5_4e4a_b2ce_d87d1dbcfbeb.slice/crio-a3e3411323b434d1d61204b819b00f04ee8d7ab3e2f4b924ad2066851971c19e WatchSource:0}: Error finding container a3e3411323b434d1d61204b819b00f04ee8d7ab3e2f4b924ad2066851971c19e: Status 404 returned error can't find the container with id a3e3411323b434d1d61204b819b00f04ee8d7ab3e2f4b924ad2066851971c19e
Jul 27 18:27:22.243920 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.243881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" event={"ID":"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb","Type":"ContainerStarted","Data":"82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5"}
Jul 27 18:27:22.244350 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.243944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" event={"ID":"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb","Type":"ContainerStarted","Data":"a3e3411323b434d1d61204b819b00f04ee8d7ab3e2f4b924ad2066851971c19e"}
Jul 27 18:27:22.244350 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.243975    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:27:22.245496 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.245474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"8b73dd00-c4d0-45d1-9de4-4d7e510c544d","Type":"ContainerStarted","Data":"6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"}
Jul 27 18:27:22.245600 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.245513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"8b73dd00-c4d0-45d1-9de4-4d7e510c544d","Type":"ContainerStarted","Data":"f7932c2de71ff3cc346ce9ff742f0474ba5ea857a142c9fb9793d5753dcdc408"}
Jul 27 18:27:22.263968 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.263913    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" podStartSLOduration=1.263900104 podStartE2EDuration="1.263900104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:27:22.262386609 +0000 UTC m=+3510.340819033" watchObservedRunningTime="2026-07-27 18:27:22.263900104 +0000 UTC m=+3510.342332528"
Jul 27 18:27:22.279993 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.279946    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podStartSLOduration=1.27993119 podStartE2EDuration="1.27993119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:27:22.278862815 +0000 UTC m=+3510.357295228" watchObservedRunningTime="2026-07-27 18:27:22.27993119 +0000 UTC m=+3510.358363683"
Jul 27 18:27:22.627158 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:22.627062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:31.294507 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:31.294470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:31.627645 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:31.627562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:31.640613 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:31.640582    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:32.302748 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:32.302711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:27:43.399558 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:43.399523    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:53.166233 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:53.166154    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:53.222371 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:53.222337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:27:53.250664 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:27:53.250634    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:28:19.239535 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:19.239496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:28:45.492415 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:45.492340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:28:47.449166 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:47.449127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:28:52.610718 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:52.610682    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:28:52.611786 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:52.611757    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:28:52.616893 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:52.616870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:28:52.620183 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:52.620163    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:28:52.621027 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:52.621007    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:28:52.625596 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:28:52.625581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:29:05.415918 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:05.415884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:05.520222 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:05.520189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:05.613521 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:05.613486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:05.713544 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:05.713515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:05.813961 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:05.813931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:08.135425 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:08.135372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:08.186871 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:08.186837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:08.717900 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:08.717871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:11.429346 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:11.429308    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:17.187055 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:17.187022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:17.254783 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:17.254750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:31.588228 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:31.588196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:31.727605 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:31.727573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:34.262386 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:34.262348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:55.323990 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:55.323952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:29:55.720546 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:29:55.720515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:30:58.020851 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:30:58.020804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:30:58.122044 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:30:58.122001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:30:58.827984 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:30:58.827937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:31:46.585401 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:31:46.585368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:31:46.655143 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:31:46.655108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:07.239566 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:07.239529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:07.340410 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:07.340375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:09.157815 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:09.157781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:09.794596 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:09.794564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:09.895864 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:09.895826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:12.004592 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:12.004561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:12.104026 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:12.103993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:12.234317 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:12.234288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:12.334957 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:12.334877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:12.898006 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:12.897969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:18.606133 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:18.606100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:18.712058 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:18.712024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:36.398418 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:36.398381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:36.611102 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:36.611067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:32:45.896643 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:32:45.896611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:33:52.639899 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:33:52.639873    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:33:52.640752 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:33:52.640719    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:33:52.644983 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:33:52.644964    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:33:52.648762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:33:52.648743    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:33:52.649585 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:33:52.649563    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:33:52.654221 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:33:52.654204    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:35:41.348892 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:35:41.348860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:36:05.300714 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:36:05.300674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:38:50.706689 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:50.706645    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:38:52.666694 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:52.666663    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:38:52.668984 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:52.667613    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:38:52.672204 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:52.672184    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:38:52.676908 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:52.676887    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:38:52.677649 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:52.677634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:38:52.681565 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:38:52.681549    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:41:40.061562 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:41:40.061526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:17.523884 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.523801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:17.547376 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.547347    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"]
Jul 27 18:42:17.554476 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.554447    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 27 18:42:17.563424 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:42:17.563396    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 18:42:17.563424 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:42:17.563424 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:42:17.563424 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb" cmd=["/bin/sleep","15"]
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.563457    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" message="" err=<
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:42:17.563470    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podUID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" containerName="main" containerID="cri-o://6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"
Jul 27 18:42:17.563631 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.563506    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podUID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" containerName="main" containerID="cri-o://6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb" gracePeriod=60
Jul 27 18:42:17.601511 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.601473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:17.916364 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.916339    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:42:17.933749 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.933723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tls-certs\") pod \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") "
Jul 27 18:42:17.933749 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.933754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-model-cache\") pod \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") "
Jul 27 18:42:17.933990 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.933777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-dshm\") pod \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") "
Jul 27 18:42:17.933990 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.933844    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jdjw\" (UniqueName: \"kubernetes.io/projected/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-kube-api-access-7jdjw\") pod \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") "
Jul 27 18:42:17.933990 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.933867    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tmp-dir\") pod \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") "
Jul 27 18:42:17.934149 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.934061    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b73dd00-c4d0-45d1-9de4-4d7e510c544d" (UID: "8b73dd00-c4d0-45d1-9de4-4d7e510c544d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:42:17.934223 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.934198    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b73dd00-c4d0-45d1-9de4-4d7e510c544d" (UID: "8b73dd00-c4d0-45d1-9de4-4d7e510c544d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:42:17.934274 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.934221    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:17.936762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.936734    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b73dd00-c4d0-45d1-9de4-4d7e510c544d" (UID: "8b73dd00-c4d0-45d1-9de4-4d7e510c544d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:42:17.936897 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.936801    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-kube-api-access-7jdjw" (OuterVolumeSpecName: "kube-api-access-7jdjw") pod "8b73dd00-c4d0-45d1-9de4-4d7e510c544d" (UID: "8b73dd00-c4d0-45d1-9de4-4d7e510c544d"). InnerVolumeSpecName "kube-api-access-7jdjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:42:17.936897 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:17.936809    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-dshm" (OuterVolumeSpecName: "dshm") pod "8b73dd00-c4d0-45d1-9de4-4d7e510c544d" (UID: "8b73dd00-c4d0-45d1-9de4-4d7e510c544d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:42:18.035326 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.035245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-home\") pod \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\" (UID: \"8b73dd00-c4d0-45d1-9de4-4d7e510c544d\") "
Jul 27 18:42:18.035516 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.035502    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jdjw\" (UniqueName: \"kubernetes.io/projected/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-kube-api-access-7jdjw\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:18.035570 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.035518    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:18.035570 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.035530    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:18.035570 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.035539    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:18.035669 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.035600    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-home" (OuterVolumeSpecName: "home") pod "8b73dd00-c4d0-45d1-9de4-4d7e510c544d" (UID: "8b73dd00-c4d0-45d1-9de4-4d7e510c544d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:42:18.136606 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.136572    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b73dd00-c4d0-45d1-9de4-4d7e510c544d-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:18.462067 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.461978    2577 generic.go:356] "Generic (PLEG): container finished" podID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" containerID="6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb" exitCode=0
Jul 27 18:42:18.462067 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.462027    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 27 18:42:18.462067 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.462034    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"8b73dd00-c4d0-45d1-9de4-4d7e510c544d","Type":"ContainerDied","Data":"6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"}
Jul 27 18:42:18.462307 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.462075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"8b73dd00-c4d0-45d1-9de4-4d7e510c544d","Type":"ContainerDied","Data":"f7932c2de71ff3cc346ce9ff742f0474ba5ea857a142c9fb9793d5753dcdc408"}
Jul 27 18:42:18.462307 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.462097    2577 scope.go:117] "RemoveContainer" containerID="6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"
Jul 27 18:42:18.472019 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.472000    2577 scope.go:117] "RemoveContainer" containerID="6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"
Jul 27 18:42:18.472257 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:42:18.472238    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb\": container with ID starting with 6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb not found: ID does not exist" containerID="6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"
Jul 27 18:42:18.472319 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.472265    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb"} err="failed to get container status \"6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb\": rpc error: code = NotFound desc = could not find container \"6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb\": container with ID starting with 6ba6834c4d912e2970a0c603e2f8481ea70c3c535cf2bcb5c4fccfee502fe8fb not found: ID does not exist"
Jul 27 18:42:18.483782 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.483756    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 27 18:42:18.485611 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:18.485590    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 27 18:42:20.482777 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:20.482743    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" path="/var/lib/kubelet/pods/8b73dd00-c4d0-45d1-9de4-4d7e510c544d/volumes"
Jul 27 18:42:23.251505 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:23.251467    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" podUID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:42:32.567118 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.567077    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" podUID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" containerName="main" containerID="cri-o://82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5" gracePeriod=45
Jul 27 18:42:32.822503 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.822444    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:42:32.874008 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.873975    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5hvf\" (UniqueName: \"kubernetes.io/projected/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-kube-api-access-n5hvf\") pod \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") "
Jul 27 18:42:32.874167 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.874103    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-tls-certs\") pod \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\" (UID: \"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb\") "
Jul 27 18:42:32.876077 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.876053    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-kube-api-access-n5hvf" (OuterVolumeSpecName: "kube-api-access-n5hvf") pod "7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" (UID: "7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb"). InnerVolumeSpecName "kube-api-access-n5hvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:42:32.876182 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.876052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" (UID: "7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:42:32.975421 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.975381    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5hvf\" (UniqueName: \"kubernetes.io/projected/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-kube-api-access-n5hvf\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:32.975421 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:32.975415    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:42:33.534378 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.534343    2577 generic.go:356] "Generic (PLEG): container finished" podID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" containerID="82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5" exitCode=0
Jul 27 18:42:33.534573 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.534391    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"
Jul 27 18:42:33.534573 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.534450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" event={"ID":"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb","Type":"ContainerDied","Data":"82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5"}
Jul 27 18:42:33.534573 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.534502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727" event={"ID":"7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb","Type":"ContainerDied","Data":"a3e3411323b434d1d61204b819b00f04ee8d7ab3e2f4b924ad2066851971c19e"}
Jul 27 18:42:33.534573 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.534523    2577 scope.go:117] "RemoveContainer" containerID="82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5"
Jul 27 18:42:33.543261 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.543242    2577 scope.go:117] "RemoveContainer" containerID="82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5"
Jul 27 18:42:33.543545 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:42:33.543526    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5\": container with ID starting with 82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5 not found: ID does not exist" containerID="82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5"
Jul 27 18:42:33.543615 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.543558    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5"} err="failed to get container status \"82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5\": rpc error: code = NotFound desc = could not find container \"82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5\": container with ID starting with 82c6cd86ba12dc0aec4f7c750f17d6f5816ffa27f0749a88033b0d3ac0f31ca5 not found: ID does not exist"
Jul 27 18:42:33.554384 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.554361    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"]
Jul 27 18:42:33.558314 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:33.558294    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-router-scheduler-5b6578b469-bk727"]
Jul 27 18:42:34.484011 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:34.483977    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" path="/var/lib/kubelet/pods/7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb/volumes"
Jul 27 18:42:38.384077 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:38.384039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:38.488966 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:38.488933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:47.042212 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.042165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:47.134833 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.134784    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 27 18:42:47.135374 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.135350    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" containerName="main"
Jul 27 18:42:47.135374 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.135376    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" containerName="main"
Jul 27 18:42:47.135595 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.135404    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" containerName="main"
Jul 27 18:42:47.135595 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.135413    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" containerName="main"
Jul 27 18:42:47.135595 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.135525    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b73dd00-c4d0-45d1-9de4-4d7e510c544d" containerName="main"
Jul 27 18:42:47.135595 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.135540    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aaa93b7-c8e5-4e4a-b2ce-d87d1dbcfbeb" containerName="main"
Jul 27 18:42:47.136630 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.136590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:47.138545 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.138528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.142597 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.142561    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"openshift-service-ca.crt\""
Jul 27 18:42:47.142597 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.142588    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"kube-root-ca.crt\""
Jul 27 18:42:47.142762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.142600    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-kserve-self-signed-certs\""
Jul 27 18:42:47.142762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.142565    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-kserve-mn-dockercfg-7dwrq\""
Jul 27 18:42:47.142762 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.142563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"default-dockercfg-bcxpf\""
Jul 27 18:42:47.152306 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.152281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 27 18:42:47.300491 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.300400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.300491 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.300468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-home\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.300733 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.300522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-dshm\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.300733 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.300554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9rj4\" (UniqueName: \"kubernetes.io/projected/8c134b65-7f15-4940-b9b2-1bf05698951d-kube-api-access-v9rj4\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.300733 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.300608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.300733 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.300668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c134b65-7f15-4940-b9b2-1bf05698951d-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.363317 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.363281    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"]
Jul 27 18:42:47.373712 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.373684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.376319 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.376297    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-epp-sa-dockercfg-n78hj\""
Jul 27 18:42:47.380010 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.379977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"]
Jul 27 18:42:47.400475 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.400443    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:42:47.401465 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401565 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c134b65-7f15-4940-b9b2-1bf05698951d-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401565 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401697 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-home\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401804 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401804 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-dshm\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401954 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9rj4\" (UniqueName: \"kubernetes.io/projected/8c134b65-7f15-4940-b9b2-1bf05698951d-kube-api-access-v9rj4\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.401954 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.401916    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-home\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.402073 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.402012    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.403854 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.403829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-dshm\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.403972 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.403958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c134b65-7f15-4940-b9b2-1bf05698951d-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.409625 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.409607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9rj4\" (UniqueName: \"kubernetes.io/projected/8c134b65-7f15-4940-b9b2-1bf05698951d-kube-api-access-v9rj4\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.449082 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.449044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:47.503084 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.503046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea671c-48aa-421a-b318-d676bedda444-tls-certs\") pod \"autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.503278 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.503107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdt7t\" (UniqueName: \"kubernetes.io/projected/01ea671c-48aa-421a-b318-d676bedda444-kube-api-access-sdt7t\") pod \"autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.589199 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.589172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 27 18:42:47.591452 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:42:47.591403    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c134b65_7f15_4940_b9b2_1bf05698951d.slice/crio-8148a744916eae59c3acf7c17a4c5ad277ae0d54dafaf0a3b54567f9ac051e2c WatchSource:0}: Error finding container 8148a744916eae59c3acf7c17a4c5ad277ae0d54dafaf0a3b54567f9ac051e2c: Status 404 returned error can't find the container with id 8148a744916eae59c3acf7c17a4c5ad277ae0d54dafaf0a3b54567f9ac051e2c
Jul 27 18:42:47.601853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.601828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"8c134b65-7f15-4940-b9b2-1bf05698951d","Type":"ContainerStarted","Data":"8148a744916eae59c3acf7c17a4c5ad277ae0d54dafaf0a3b54567f9ac051e2c"}
Jul 27 18:42:47.604239 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.604211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea671c-48aa-421a-b318-d676bedda444-tls-certs\") pod \"autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.604359 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.604324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdt7t\" (UniqueName: \"kubernetes.io/projected/01ea671c-48aa-421a-b318-d676bedda444-kube-api-access-sdt7t\") pod \"autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.606715 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.606689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea671c-48aa-421a-b318-d676bedda444-tls-certs\") pod \"autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.613045 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.613025    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdt7t\" (UniqueName: \"kubernetes.io/projected/01ea671c-48aa-421a-b318-d676bedda444-kube-api-access-sdt7t\") pod \"autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.689577 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.689545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:47.827290 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:47.827253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"]
Jul 27 18:42:47.829443 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:42:47.829401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ea671c_48aa_421a_b318_d676bedda444.slice/crio-51130d1c400e9e36d8b987cb0fbd158b9a362b35524141c655ea9ed735251960 WatchSource:0}: Error finding container 51130d1c400e9e36d8b987cb0fbd158b9a362b35524141c655ea9ed735251960: Status 404 returned error can't find the container with id 51130d1c400e9e36d8b987cb0fbd158b9a362b35524141c655ea9ed735251960
Jul 27 18:42:48.609061 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:48.609020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" event={"ID":"01ea671c-48aa-421a-b318-d676bedda444","Type":"ContainerStarted","Data":"cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e"}
Jul 27 18:42:48.609533 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:48.609084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" event={"ID":"01ea671c-48aa-421a-b318-d676bedda444","Type":"ContainerStarted","Data":"51130d1c400e9e36d8b987cb0fbd158b9a362b35524141c655ea9ed735251960"}
Jul 27 18:42:48.609533 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:48.609110    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:42:48.610536 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:48.610515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"8c134b65-7f15-4940-b9b2-1bf05698951d","Type":"ContainerStarted","Data":"ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"}
Jul 27 18:42:48.625686 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:48.625639    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" podStartSLOduration=1.6256238939999998 podStartE2EDuration="1.625623894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:42:48.625418547 +0000 UTC m=+4436.703850980" watchObservedRunningTime="2026-07-27 18:42:48.625623894 +0000 UTC m=+4436.704056316"
Jul 27 18:42:48.642439 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:48.642374    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podStartSLOduration=1.6423598529999999 podStartE2EDuration="1.642359853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:42:48.640293276 +0000 UTC m=+4436.718725701" watchObservedRunningTime="2026-07-27 18:42:48.642359853 +0000 UTC m=+4436.720792276"
Jul 27 18:42:49.449573 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:49.449534    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:57.449630 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:57.449591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:57.462009 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:57.461982    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:42:57.664903 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:42:57.664870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:43:07.135960 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:07.135925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:43:19.616977 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:19.616948    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:43:52.694616 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:52.694510    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:43:52.698689 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:52.695534    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:43:52.700589 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:52.700566    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:43:52.706921 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:52.706905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:43:52.707709 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:52.707696    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:43:52.711980 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:43:52.711963    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:44:40.171819 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:44:40.171781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:44:40.226217 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:44:40.226184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:45:51.408453 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:45:51.408400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:06.227745 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:06.227707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:29.609472 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:29.609424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:29.708549 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:29.708512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:29.797916 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:29.797882    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:29.862309 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:29.862216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:32.435378 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:32.435344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:32.499413 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:32.499384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:47:32.895894 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:47:32.895860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:48:23.935264 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:23.935173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:48:24.023152 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.023118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:48:24.078221 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.078185    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 27 18:48:24.081623 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.081597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.085614 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.085586    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"kube-root-ca.crt\""
Jul 27 18:48:24.085614 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.085604    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"openshift-service-ca.crt\""
Jul 27 18:48:24.085614 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.085586    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs\""
Jul 27 18:48:24.085945 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.085929    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"default-dockercfg-g5bb7\""
Jul 27 18:48:24.086037 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.086016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"llmisvc-model-pvc-router-manage-2577e7-b09f84b1-dockercfg-s2b5n\""
Jul 27 18:48:24.091817 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.091793    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 27 18:48:24.105717 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.105688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:48:24.168310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.168034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.168310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.168100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.168310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.168153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.168310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.168303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.168668 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.168352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.168668 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.168408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5wl\" (UniqueName: \"kubernetes.io/projected/9029a2a7-6c8f-4f5a-b988-53345fa8df82-kube-api-access-fb5wl\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.211283 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.211192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:48:24.269806 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.269767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270008 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.269823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270008 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.269880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270008 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.269913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270008 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.269965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5wl\" (UniqueName: \"kubernetes.io/projected/9029a2a7-6c8f-4f5a-b988-53345fa8df82-kube-api-access-fb5wl\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270008 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.270006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270293 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.270137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270293 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.270197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.270293 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.270266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.272179 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.272151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.272386 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.272365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:24.281165 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:24.281142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5wl\" (UniqueName: \"kubernetes.io/projected/9029a2a7-6c8f-4f5a-b988-53345fa8df82-kube-api-access-fb5wl\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:48:50.257507 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:50.257465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:48:52.721973 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:52.721946    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:48:52.724231 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:52.722878    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:48:52.727940 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:52.727919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:48:52.735652 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:52.735631    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:48:52.736557 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:52.736541    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:48:52.741009 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:48:52.740991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:49:46.382263 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:49:46.382216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:12.145111 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:12.145074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:27.093516 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:50:27.093472    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" podUID="9029a2a7-6c8f-4f5a-b988-53345fa8df82"
Jul 27 18:50:27.770099 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:27.770067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:50:31.693209 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:31.693174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:31.798319 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:31.798289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:47.840544 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.840506    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 27 18:50:47.841116 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:50:47.840889    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context canceled" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1" podUID="9029a2a7-6c8f-4f5a-b988-53345fa8df82"
Jul 27 18:50:47.861851 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.861819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:50:47.868482 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.868461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:50:47.995478 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995424    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-home\") pod \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") "
Jul 27 18:50:47.995478 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995475    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tmp-dir\") pod \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") "
Jul 27 18:50:47.995700 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-dshm\") pod \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") "
Jul 27 18:50:47.995700 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995532    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-model-cache\") pod \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") "
Jul 27 18:50:47.995700 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb5wl\" (UniqueName: \"kubernetes.io/projected/9029a2a7-6c8f-4f5a-b988-53345fa8df82-kube-api-access-fb5wl\") pod \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") "
Jul 27 18:50:47.995700 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995615    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tls-certs\") pod \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\" (UID: \"9029a2a7-6c8f-4f5a-b988-53345fa8df82\") "
Jul 27 18:50:47.995700 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995690    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-home" (OuterVolumeSpecName: "home") pod "9029a2a7-6c8f-4f5a-b988-53345fa8df82" (UID: "9029a2a7-6c8f-4f5a-b988-53345fa8df82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:50:47.995968 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9029a2a7-6c8f-4f5a-b988-53345fa8df82" (UID: "9029a2a7-6c8f-4f5a-b988-53345fa8df82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:50:47.995968 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995812    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-model-cache" (OuterVolumeSpecName: "model-cache") pod "9029a2a7-6c8f-4f5a-b988-53345fa8df82" (UID: "9029a2a7-6c8f-4f5a-b988-53345fa8df82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:50:47.995968 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995941    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:50:47.995968 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995955    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:50:47.995968 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.995968    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:50:47.997804 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.997780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-dshm" (OuterVolumeSpecName: "dshm") pod "9029a2a7-6c8f-4f5a-b988-53345fa8df82" (UID: "9029a2a7-6c8f-4f5a-b988-53345fa8df82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:50:47.997804 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.997787    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9029a2a7-6c8f-4f5a-b988-53345fa8df82" (UID: "9029a2a7-6c8f-4f5a-b988-53345fa8df82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:50:47.997939 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:47.997802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9029a2a7-6c8f-4f5a-b988-53345fa8df82-kube-api-access-fb5wl" (OuterVolumeSpecName: "kube-api-access-fb5wl") pod "9029a2a7-6c8f-4f5a-b988-53345fa8df82" (UID: "9029a2a7-6c8f-4f5a-b988-53345fa8df82"). InnerVolumeSpecName "kube-api-access-fb5wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:50:48.097258 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:48.097185    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9029a2a7-6c8f-4f5a-b988-53345fa8df82-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:50:48.097258 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:48.097211    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb5wl\" (UniqueName: \"kubernetes.io/projected/9029a2a7-6c8f-4f5a-b988-53345fa8df82-kube-api-access-fb5wl\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:50:48.097258 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:48.097221    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9029a2a7-6c8f-4f5a-b988-53345fa8df82-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:50:48.866089 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:48.866007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"
Jul 27 18:50:48.893738 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:48.893706    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 27 18:50:48.897293 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:48.897265    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1"]
Jul 27 18:50:50.483644 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:50.483609    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9029a2a7-6c8f-4f5a-b988-53345fa8df82" path="/var/lib/kubelet/pods/9029a2a7-6c8f-4f5a-b988-53345fa8df82/volumes"
Jul 27 18:50:51.375553 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:51.375513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:51.475886 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:51.475854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:51.578035 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:51.578002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:53.802207 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:53.802172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:53.903538 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:53.903498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:54.002685 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:54.002650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:56.895980 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:56.895943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:56.995925 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:56.995889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:57.838685 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:57.838654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:59.616083 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:59.616050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:50:59.671651 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:50:59.671617    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:03.475116 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:03.475083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:03.588926 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:03.588890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:04.141488 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:04.141456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:04.206981 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:04.206948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:09.417352 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:09.417316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:09.673554 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:09.673472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:19.120305 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:19.120228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:19.220088 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:19.220054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:21.017399 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:21.017360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:21.225741 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:21.225708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:21.326041 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:21.325964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:25.440615 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:25.440577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:25.541033 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:25.541002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:25.645563 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:25.645535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:26.448169 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:26.448133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:30.408275 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:30.408234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:30.660598 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:30.660514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:30.764650 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:30.764614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:31.599420 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:31.599381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:31.699885 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:31.699853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:35.609338 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:35.609302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:35.709636 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:35.709604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:35.808560 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:35.808526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:36.908318 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:36.908289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:37.008273 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:37.008239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:37.112312 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:37.112282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:37.209349 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:37.209264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:37.699376 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:37.699344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:37.801289 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:37.801258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:38.703599 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:38.703564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:38.804203 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:38.804171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:38.904128 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:38.904097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:39.594251 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:39.594215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:39.694741 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:39.694680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:39.797873 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:39.797838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:40.867877 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:40.867846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:40.964362 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:40.964326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:42.272296 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:42.272265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:42.375202 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:42.375159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:43.597581 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:43.597546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:43.697957 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:43.697922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:43.800898 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:43.800858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:43.901316 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:43.901218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:44.677601 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:44.677568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:44.785047 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:44.785015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:45.521260 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:45.521222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:45.623405 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:45.623370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:46.866150 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:46.866117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:51:46.966304 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:51:46.966273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:08.109187 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.109155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:08.267401 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.267370    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"]
Jul 27 18:52:08.270914 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.270892    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.274669 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.274641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Jul 27 18:52:08.274775 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.274686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 27 18:52:08.274775 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.274740    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-jrx85\""
Jul 27 18:52:08.274889 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.274782    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Jul 27 18:52:08.274889 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.274826    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-wt2gj\""
Jul 27 18:52:08.279522 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.279505    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"]
Jul 27 18:52:08.326499 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.326468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.326661 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.326588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c426z\" (UniqueName: \"kubernetes.io/projected/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-kube-api-access-c426z\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.427651 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.427560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c426z\" (UniqueName: \"kubernetes.io/projected/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-kube-api-access-c426z\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.427651 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.427625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.430234 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.430208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.435804 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.435783    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c426z\" (UniqueName: \"kubernetes.io/projected/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-kube-api-access-c426z\") pod \"stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.583106 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.583073    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:08.718846 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:08.718823    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"]
Jul 27 18:52:08.721215 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:52:08.721188    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a6a722_6e47_4eaf_919e_c7b3ed43bd52.slice/crio-35c03066da82c6300d83e2f31e606ce5e9e2b7f229b48b4dd51d5e0512795707 WatchSource:0}: Error finding container 35c03066da82c6300d83e2f31e606ce5e9e2b7f229b48b4dd51d5e0512795707: Status 404 returned error can't find the container with id 35c03066da82c6300d83e2f31e606ce5e9e2b7f229b48b4dd51d5e0512795707
Jul 27 18:52:09.241472 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:09.241418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" event={"ID":"45a6a722-6e47-4eaf-919e-c7b3ed43bd52","Type":"ContainerStarted","Data":"dca70236f400a1a65e1e0a0fbb3a675de626477bf22b4ff1174ceace1926e46e"}
Jul 27 18:52:09.241906 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:09.241498    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:52:09.241906 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:09.241516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" event={"ID":"45a6a722-6e47-4eaf-919e-c7b3ed43bd52","Type":"ContainerStarted","Data":"35c03066da82c6300d83e2f31e606ce5e9e2b7f229b48b4dd51d5e0512795707"}
Jul 27 18:52:09.261381 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:09.261318    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" podStartSLOduration=1.261296371 podStartE2EDuration="1.261296371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:52:09.258868015 +0000 UTC m=+4997.337300439" watchObservedRunningTime="2026-07-27 18:52:09.261296371 +0000 UTC m=+4997.339728797"
Jul 27 18:52:09.588881 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:09.588796    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:34.802523 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:34.802486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:35.845128 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:35.845094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:38.293925 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:38.293896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:38.487200 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:38.487167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:52:40.248900 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:52:40.248871    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:53:52.750040 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:53:52.749945    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:53:52.753285 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:53:52.750935    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:53:52.756009 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:53:52.755988    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:53:52.763479 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:53:52.763460    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:53:52.764302 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:53:52.764287    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:53:52.768843 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:53:52.768828    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:54:16.016960 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:54:16.016863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:54:27.606227 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:54:27.606193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:55:06.477807 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:55:06.477768    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-jrx85\" not found"
Jul 27 18:55:36.483139 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:55:36.483103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:56:27.477419 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:27.477387    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" secret="" err="secret \"stop-feature-test-epp-sa-dockercfg-jrx85\" not found"
Jul 27 18:56:49.482465 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:49.482400    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"]
Jul 27 18:56:49.486197 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:49.486158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:56:49.545967 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:49.545924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:56:50.249804 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:50.249763    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:56:54.632863 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:54.632825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:56:54.695165 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:56:54.695127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:00.249606 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:00.249565    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:57:04.506786 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.506736    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerName="main" containerID="cri-o://dca70236f400a1a65e1e0a0fbb3a675de626477bf22b4ff1174ceace1926e46e" gracePeriod=45
Jul 27 18:57:04.626576 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.626541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:04.628984 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.628961    2577 generic.go:356] "Generic (PLEG): container finished" podID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerID="dca70236f400a1a65e1e0a0fbb3a675de626477bf22b4ff1174ceace1926e46e" exitCode=0
Jul 27 18:57:04.629095 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.628997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" event={"ID":"45a6a722-6e47-4eaf-919e-c7b3ed43bd52","Type":"ContainerDied","Data":"dca70236f400a1a65e1e0a0fbb3a675de626477bf22b4ff1174ceace1926e46e"}
Jul 27 18:57:04.760305 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.760238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:57:04.893578 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.893545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c426z\" (UniqueName: \"kubernetes.io/projected/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-kube-api-access-c426z\") pod \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") "
Jul 27 18:57:04.893743 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.893631    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-tls-certs\") pod \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\" (UID: \"45a6a722-6e47-4eaf-919e-c7b3ed43bd52\") "
Jul 27 18:57:04.895763 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.895733    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45a6a722-6e47-4eaf-919e-c7b3ed43bd52" (UID: "45a6a722-6e47-4eaf-919e-c7b3ed43bd52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:57:04.895881 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.895775    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-kube-api-access-c426z" (OuterVolumeSpecName: "kube-api-access-c426z") pod "45a6a722-6e47-4eaf-919e-c7b3ed43bd52" (UID: "45a6a722-6e47-4eaf-919e-c7b3ed43bd52"). InnerVolumeSpecName "kube-api-access-c426z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:57:04.994679 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.994647    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c426z\" (UniqueName: \"kubernetes.io/projected/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-kube-api-access-c426z\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:04.994679 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:04.994672    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45a6a722-6e47-4eaf-919e-c7b3ed43bd52-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:05.642014 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:05.641979    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"
Jul 27 18:57:05.642537 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:05.641976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh" event={"ID":"45a6a722-6e47-4eaf-919e-c7b3ed43bd52","Type":"ContainerDied","Data":"35c03066da82c6300d83e2f31e606ce5e9e2b7f229b48b4dd51d5e0512795707"}
Jul 27 18:57:05.642537 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:05.642127    2577 scope.go:117] "RemoveContainer" containerID="dca70236f400a1a65e1e0a0fbb3a675de626477bf22b4ff1174ceace1926e46e"
Jul 27 18:57:05.663278 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:05.663250    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"]
Jul 27 18:57:05.668509 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:05.668481    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6ff9c88bb8-qbvzh"]
Jul 27 18:57:06.483680 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:06.483646    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" path="/var/lib/kubelet/pods/45a6a722-6e47-4eaf-919e-c7b3ed43bd52/volumes"
Jul 27 18:57:08.505050 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:08.505012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:08.602757 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:08.602715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:08.703181 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:08.703145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:14.821902 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:14.821821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:20.494221 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:20.494183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:20.595117 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:20.595088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:27.514822 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:27.514786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:53.449566 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.449473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:53.454412 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.454385    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"]
Jul 27 18:57:53.459599 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.459572    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 27 18:57:53.468198 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:57:53.468161    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 18:57:53.468198 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:57:53.468198 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:57:53.468198 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613" cmd=["/bin/sleep","15"]
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.468220    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" message="" err=<
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:57:53.468236    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerName="main" containerID="cri-o://ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"
Jul 27 18:57:53.468451 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.468278    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerName="main" containerID="cri-o://ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613" gracePeriod=60
Jul 27 18:57:53.549440 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.549397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:57:53.654727 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.654676    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/health\": dial tcp 10.134.0.51:8000: connect: connection refused"
Jul 27 18:57:53.835720 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.835695    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:57:53.865060 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.865028    2577 generic.go:356] "Generic (PLEG): container finished" podID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerID="ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613" exitCode=0
Jul 27 18:57:53.865222 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.865089    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 27 18:57:53.865222 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.865119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"8c134b65-7f15-4940-b9b2-1bf05698951d","Type":"ContainerDied","Data":"ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"}
Jul 27 18:57:53.865222 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.865200    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"8c134b65-7f15-4940-b9b2-1bf05698951d","Type":"ContainerDied","Data":"8148a744916eae59c3acf7c17a4c5ad277ae0d54dafaf0a3b54567f9ac051e2c"}
Jul 27 18:57:53.865327 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.865225    2577 scope.go:117] "RemoveContainer" containerID="ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"
Jul 27 18:57:53.874200 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.874182    2577 scope.go:117] "RemoveContainer" containerID="ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"
Jul 27 18:57:53.874488 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:57:53.874468    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613\": container with ID starting with ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613 not found: ID does not exist" containerID="ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"
Jul 27 18:57:53.874572 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.874496    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613"} err="failed to get container status \"ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613\": rpc error: code = NotFound desc = could not find container \"ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613\": container with ID starting with ad6783db584d9af4e529467291f3a774548049860afa19f13785082fd1071613 not found: ID does not exist"
Jul 27 18:57:53.927029 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.926994    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9rj4\" (UniqueName: \"kubernetes.io/projected/8c134b65-7f15-4940-b9b2-1bf05698951d-kube-api-access-v9rj4\") pod \"8c134b65-7f15-4940-b9b2-1bf05698951d\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") "
Jul 27 18:57:53.927029 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-tmp-dir\") pod \"8c134b65-7f15-4940-b9b2-1bf05698951d\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") "
Jul 27 18:57:53.927234 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-home\") pod \"8c134b65-7f15-4940-b9b2-1bf05698951d\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") "
Jul 27 18:57:53.927234 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-dshm\") pod \"8c134b65-7f15-4940-b9b2-1bf05698951d\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") "
Jul 27 18:57:53.927234 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927160    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-model-cache\") pod \"8c134b65-7f15-4940-b9b2-1bf05698951d\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") "
Jul 27 18:57:53.927234 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927192    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c134b65-7f15-4940-b9b2-1bf05698951d-tls-certs\") pod \"8c134b65-7f15-4940-b9b2-1bf05698951d\" (UID: \"8c134b65-7f15-4940-b9b2-1bf05698951d\") "
Jul 27 18:57:53.927413 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927308    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c134b65-7f15-4940-b9b2-1bf05698951d" (UID: "8c134b65-7f15-4940-b9b2-1bf05698951d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:57:53.927413 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927359    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-home" (OuterVolumeSpecName: "home") pod "8c134b65-7f15-4940-b9b2-1bf05698951d" (UID: "8c134b65-7f15-4940-b9b2-1bf05698951d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:57:53.927569 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927474    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c134b65-7f15-4940-b9b2-1bf05698951d" (UID: "8c134b65-7f15-4940-b9b2-1bf05698951d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:57:53.927569 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927548    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:53.927569 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.927563    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:53.929409 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.929388    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-dshm" (OuterVolumeSpecName: "dshm") pod "8c134b65-7f15-4940-b9b2-1bf05698951d" (UID: "8c134b65-7f15-4940-b9b2-1bf05698951d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 18:57:53.929558 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.929446    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c134b65-7f15-4940-b9b2-1bf05698951d-kube-api-access-v9rj4" (OuterVolumeSpecName: "kube-api-access-v9rj4") pod "8c134b65-7f15-4940-b9b2-1bf05698951d" (UID: "8c134b65-7f15-4940-b9b2-1bf05698951d"). InnerVolumeSpecName "kube-api-access-v9rj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:57:53.929558 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:53.929480    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c134b65-7f15-4940-b9b2-1bf05698951d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c134b65-7f15-4940-b9b2-1bf05698951d" (UID: "8c134b65-7f15-4940-b9b2-1bf05698951d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:57:54.028333 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.028245    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9rj4\" (UniqueName: \"kubernetes.io/projected/8c134b65-7f15-4940-b9b2-1bf05698951d-kube-api-access-v9rj4\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:54.028333 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.028276    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:54.028333 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.028287    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c134b65-7f15-4940-b9b2-1bf05698951d-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:54.028333 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.028296    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c134b65-7f15-4940-b9b2-1bf05698951d-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:57:54.189201 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.189173    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 27 18:57:54.195381 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.195357    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 27 18:57:54.483195 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:54.483158    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" path="/var/lib/kubelet/pods/8c134b65-7f15-4940-b9b2-1bf05698951d/volumes"
Jul 27 18:57:59.617286 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:57:59.617244    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" podUID="01ea671c-48aa-421a-b318-d676bedda444" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 18:58:08.474292 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.474245    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" podUID="01ea671c-48aa-421a-b318-d676bedda444" containerName="main" containerID="cri-o://cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e" gracePeriod=45
Jul 27 18:58:08.735061 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.734994    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:58:08.864176 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.864135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea671c-48aa-421a-b318-d676bedda444-tls-certs\") pod \"01ea671c-48aa-421a-b318-d676bedda444\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") "
Jul 27 18:58:08.864363 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.864234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdt7t\" (UniqueName: \"kubernetes.io/projected/01ea671c-48aa-421a-b318-d676bedda444-kube-api-access-sdt7t\") pod \"01ea671c-48aa-421a-b318-d676bedda444\" (UID: \"01ea671c-48aa-421a-b318-d676bedda444\") "
Jul 27 18:58:08.866327 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.866298    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ea671c-48aa-421a-b318-d676bedda444-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01ea671c-48aa-421a-b318-d676bedda444" (UID: "01ea671c-48aa-421a-b318-d676bedda444"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 18:58:08.866327 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.866312    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01ea671c-48aa-421a-b318-d676bedda444-kube-api-access-sdt7t" (OuterVolumeSpecName: "kube-api-access-sdt7t") pod "01ea671c-48aa-421a-b318-d676bedda444" (UID: "01ea671c-48aa-421a-b318-d676bedda444"). InnerVolumeSpecName "kube-api-access-sdt7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 18:58:08.938565 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.938531    2577 generic.go:356] "Generic (PLEG): container finished" podID="01ea671c-48aa-421a-b318-d676bedda444" containerID="cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e" exitCode=0
Jul 27 18:58:08.938781 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.938577    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"
Jul 27 18:58:08.938781 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.938595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" event={"ID":"01ea671c-48aa-421a-b318-d676bedda444","Type":"ContainerDied","Data":"cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e"}
Jul 27 18:58:08.938781 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.938636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228" event={"ID":"01ea671c-48aa-421a-b318-d676bedda444","Type":"ContainerDied","Data":"51130d1c400e9e36d8b987cb0fbd158b9a362b35524141c655ea9ed735251960"}
Jul 27 18:58:08.938781 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.938658    2577 scope.go:117] "RemoveContainer" containerID="cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e"
Jul 27 18:58:08.947895 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.947877    2577 scope.go:117] "RemoveContainer" containerID="cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e"
Jul 27 18:58:08.948141 ip-10-0-135-40 kubenswrapper[2577]: E0727 18:58:08.948123    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e\": container with ID starting with cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e not found: ID does not exist" containerID="cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e"
Jul 27 18:58:08.948193 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.948148    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e"} err="failed to get container status \"cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e\": rpc error: code = NotFound desc = could not find container \"cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e\": container with ID starting with cb19f18aae9ee1faf43a02dd9beb05fe0363e1adc01fa7e93eb269dc4c39097e not found: ID does not exist"
Jul 27 18:58:08.959617 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.959593    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"]
Jul 27 18:58:08.964731 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.964704    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-router-scheduler-55c6bcf4c9-t4228"]
Jul 27 18:58:08.965104 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.965083    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdt7t\" (UniqueName: \"kubernetes.io/projected/01ea671c-48aa-421a-b318-d676bedda444-kube-api-access-sdt7t\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:58:08.965171 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:08.965112    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea671c-48aa-421a-b318-d676bedda444-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 18:58:10.467746 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:10.467710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:58:10.483419 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:10.483387    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01ea671c-48aa-421a-b318-d676bedda444" path="/var/lib/kubelet/pods/01ea671c-48aa-421a-b318-d676bedda444/volumes"
Jul 27 18:58:10.565490 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:10.565457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:58:32.400359 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400328    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"]
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400691    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400702    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400712    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400718    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400739    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ea671c-48aa-421a-b318-d676bedda444" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400745    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ea671c-48aa-421a-b318-d676bedda444" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400805    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a6a722-6e47-4eaf-919e-c7b3ed43bd52" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400812    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ea671c-48aa-421a-b318-d676bedda444" containerName="main"
Jul 27 18:58:32.400853 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.400820    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c134b65-7f15-4940-b9b2-1bf05698951d" containerName="main"
Jul 27 18:58:32.403756 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.403739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.407339 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.407316    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"default-dockercfg-jp964\""
Jul 27 18:58:32.408307 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.408290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"autoscale-cleanup-hpa-kserve-self-signed-certs\""
Jul 27 18:58:32.408403 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.408295    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"kube-root-ca.crt\""
Jul 27 18:58:32.408403 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.408359    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"openshift-service-ca.crt\""
Jul 27 18:58:32.414984 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.414964    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"]
Jul 27 18:58:32.425471 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.425449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:58:32.471005 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.470966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-dshm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.471170 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.471019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8snr\" (UniqueName: \"kubernetes.io/projected/591c360d-cd22-4518-adbc-674139848e5a-kube-api-access-n8snr\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.471170 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.471079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-model-cache\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.471170 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.471130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-tmp-dir\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.471170 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.471148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-home\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.471321 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.471180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/591c360d-cd22-4518-adbc-674139848e5a-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572036 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.571988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-tmp-dir\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572036 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-home\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/591c360d-cd22-4518-adbc-674139848e5a-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-dshm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572310 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8snr\" (UniqueName: \"kubernetes.io/projected/591c360d-cd22-4518-adbc-674139848e5a-kube-api-access-n8snr\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572536 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-model-cache\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572623 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-home\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572623 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572518    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-tmp-dir\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.572778 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.572758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-model-cache\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.574599 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.574571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-dshm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.574886 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.574867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/591c360d-cd22-4518-adbc-674139848e5a-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.580361 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.580341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8snr\" (UniqueName: \"kubernetes.io/projected/591c360d-cd22-4518-adbc-674139848e5a-kube-api-access-n8snr\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.617593 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.617559    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"]
Jul 27 18:58:32.621567 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.621539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.623963 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.623944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"autoscale-cleanup-hpa-epp-sa-dockercfg-d6cwr\""
Jul 27 18:58:32.629154 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.629131    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"]
Jul 27 18:58:32.644276 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.644245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:58:32.673418 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.673348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f13d3d-54b3-4d96-acde-f36c3da56da2-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.673418 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.673409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2rrv\" (UniqueName: \"kubernetes.io/projected/17f13d3d-54b3-4d96-acde-f36c3da56da2-kube-api-access-b2rrv\") pod \"autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.714480 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.714451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:32.774784 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.774753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2rrv\" (UniqueName: \"kubernetes.io/projected/17f13d3d-54b3-4d96-acde-f36c3da56da2-kube-api-access-b2rrv\") pod \"autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.774939 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.774841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f13d3d-54b3-4d96-acde-f36c3da56da2-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.777570 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.777543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f13d3d-54b3-4d96-acde-f36c3da56da2-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.784256 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.784232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2rrv\" (UniqueName: \"kubernetes.io/projected/17f13d3d-54b3-4d96-acde-f36c3da56da2-kube-api-access-b2rrv\") pod \"autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:32.850819 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.850791    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"]
Jul 27 18:58:32.854063 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:58:32.854035    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591c360d_cd22_4518_adbc_674139848e5a.slice/crio-12ff9ac424efa64dd05742ffe7173c6dbab1be2313a99576315421323778f00d WatchSource:0}: Error finding container 12ff9ac424efa64dd05742ffe7173c6dbab1be2313a99576315421323778f00d: Status 404 returned error can't find the container with id 12ff9ac424efa64dd05742ffe7173c6dbab1be2313a99576315421323778f00d
Jul 27 18:58:32.933535 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:32.933459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:33.050950 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:33.050909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" event={"ID":"591c360d-cd22-4518-adbc-674139848e5a","Type":"ContainerStarted","Data":"8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f"}
Jul 27 18:58:33.051149 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:33.050969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" event={"ID":"591c360d-cd22-4518-adbc-674139848e5a","Type":"ContainerStarted","Data":"12ff9ac424efa64dd05742ffe7173c6dbab1be2313a99576315421323778f00d"}
Jul 27 18:58:33.062805 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:33.062781    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"]
Jul 27 18:58:33.065853 ip-10-0-135-40 kubenswrapper[2577]: W0727 18:58:33.065826    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f13d3d_54b3_4d96_acde_f36c3da56da2.slice/crio-e2eaaa75694ca9e44dcd3efe5ffb04a7583e92f830f720d1d315dee0e01156e8 WatchSource:0}: Error finding container e2eaaa75694ca9e44dcd3efe5ffb04a7583e92f830f720d1d315dee0e01156e8: Status 404 returned error can't find the container with id e2eaaa75694ca9e44dcd3efe5ffb04a7583e92f830f720d1d315dee0e01156e8
Jul 27 18:58:33.066680 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:33.066636    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" podStartSLOduration=1.066619613 podStartE2EDuration="1.066619613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:58:33.065540539 +0000 UTC m=+5381.143972962" watchObservedRunningTime="2026-07-27 18:58:33.066619613 +0000 UTC m=+5381.145052038"
Jul 27 18:58:33.715034 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:33.715000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:34.057059 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:34.056965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" event={"ID":"17f13d3d-54b3-4d96-acde-f36c3da56da2","Type":"ContainerStarted","Data":"faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894"}
Jul 27 18:58:34.057059 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:34.057027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" event={"ID":"17f13d3d-54b3-4d96-acde-f36c3da56da2","Type":"ContainerStarted","Data":"e2eaaa75694ca9e44dcd3efe5ffb04a7583e92f830f720d1d315dee0e01156e8"}
Jul 27 18:58:34.074538 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:34.074488    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" podStartSLOduration=2.074472921 podStartE2EDuration="2.074472921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 18:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 18:58:34.072930504 +0000 UTC m=+5382.151362928" watchObservedRunningTime="2026-07-27 18:58:34.074472921 +0000 UTC m=+5382.152905345"
Jul 27 18:58:35.061113 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:35.061078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:58:42.715035 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:42.715005    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:42.727376 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:42.727350    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:43.109846 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:43.109761    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 18:58:52.778137 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:52.778028    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:58:52.798868 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:52.779057    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:58:52.798868 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:52.784112    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:58:52.798868 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:52.791978    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 18:58:52.798868 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:52.792758    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 18:58:52.798868 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:52.797249    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 18:58:53.826199 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:58:53.826163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:59:06.067353 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:59:06.067324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 18:59:12.743883 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:59:12.743848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:59:12.794108 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:59:12.794079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 18:59:41.106038 ip-10-0-135-40 kubenswrapper[2577]: I0727 18:59:41.105998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:00:02.705640 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:00:02.705607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:00:02.814140 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:00:02.814101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:00:02.909952 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:00:02.909892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:00:08.466470 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:00:08.466400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:00:08.568094 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:00:08.568059    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:00:22.084145 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:00:22.084051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:01:57.188158 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:01:57.188108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:02:25.500044 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:02:25.500011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:02:53.036130 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:02:53.036100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:10.825493 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:10.825452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:10.925105 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:10.925065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:16.887961 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:16.887879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:16.945410 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:16.945377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:50.714525 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:50.714488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:50.930974 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:50.930937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:03:52.804440 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:52.804324    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:03:52.808481 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:52.805386    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:03:52.810471 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:52.810451    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 19:03:52.819873 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:52.819855    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:03:52.823320 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:52.823303    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:03:52.828074 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:03:52.828058    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 19:04:11.992449 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:04:11.992402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:04:15.199634 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:04:15.199595    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:04:15.302379 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:04:15.302344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:16.469130 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:16.469052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:16.527233 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:16.527192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:31.226075 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:31.226041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:35.377174 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:35.377136    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:35.474553 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:35.474516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:36.215035 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:36.214992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:36.312487 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:36.312454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:41.419165 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:41.419123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:44.783416 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:44.783388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:45.020830 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.020776    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"]
Jul 27 19:06:45.024376 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.024360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.028105 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.028076    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Jul 27 19:06:45.028105 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.028100    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-6685r\""
Jul 27 19:06:45.028272 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.028109    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-epp-sa-dockercfg-kgd8m\""
Jul 27 19:06:45.028272 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.028208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Jul 27 19:06:45.028343 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.028333    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Jul 27 19:06:45.033634 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.033574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"]
Jul 27 19:06:45.059403 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.059372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:45.135858 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.135828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdmzz\" (UniqueName: \"kubernetes.io/projected/76010446-b3c2-4abc-863d-194ddde959bb-kube-api-access-gdmzz\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.136000 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.135862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76010446-b3c2-4abc-863d-194ddde959bb-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.236417 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.236381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdmzz\" (UniqueName: \"kubernetes.io/projected/76010446-b3c2-4abc-863d-194ddde959bb-kube-api-access-gdmzz\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.236417 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.236420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76010446-b3c2-4abc-863d-194ddde959bb-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.238947 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.238919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76010446-b3c2-4abc-863d-194ddde959bb-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.243887 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.243870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdmzz\" (UniqueName: \"kubernetes.io/projected/76010446-b3c2-4abc-863d-194ddde959bb-kube-api-access-gdmzz\") pod \"prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.337285 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.337194    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:45.469167 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:45.469121    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"]
Jul 27 19:06:45.471499 ip-10-0-135-40 kubenswrapper[2577]: W0727 19:06:45.471471    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76010446_b3c2_4abc_863d_194ddde959bb.slice/crio-d2d0df28a1ae6f0d5e90fc21e8cad811cf83dad31a378a996add915f9b03ded3 WatchSource:0}: Error finding container d2d0df28a1ae6f0d5e90fc21e8cad811cf83dad31a378a996add915f9b03ded3: Status 404 returned error can't find the container with id d2d0df28a1ae6f0d5e90fc21e8cad811cf83dad31a378a996add915f9b03ded3
Jul 27 19:06:46.322365 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:46.322331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" event={"ID":"76010446-b3c2-4abc-863d-194ddde959bb","Type":"ContainerStarted","Data":"3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c"}
Jul 27 19:06:46.322821 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:46.322381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" event={"ID":"76010446-b3c2-4abc-863d-194ddde959bb","Type":"ContainerStarted","Data":"d2d0df28a1ae6f0d5e90fc21e8cad811cf83dad31a378a996add915f9b03ded3"}
Jul 27 19:06:46.322821 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:46.322518    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:06:46.339492 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:46.339420    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" podStartSLOduration=1.339401676 podStartE2EDuration="1.339401676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:06:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:06:46.337234164 +0000 UTC m=+5874.415666588" watchObservedRunningTime="2026-07-27 19:06:46.339401676 +0000 UTC m=+5874.417834098"
Jul 27 19:06:47.073268 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:47.073231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:06:47.172130 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:06:47.172101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:07:05.716144 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:07:05.716103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:07:17.329509 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:07:17.329477    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:08:30.492702 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:30.492666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:08:52.832673 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:52.832567    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:08:52.836594 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:52.833575    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:08:52.838806 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:52.838784    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 19:08:52.853051 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:52.853022    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:08:52.853977 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:52.853956    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:08:52.858475 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:08:52.858459    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 19:09:12.796543 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:12.796505    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"]
Jul 27 19:09:12.812207 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:12.812179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:09:12.911559 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:12.911517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:09:17.329357 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:17.329274    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" podUID="76010446-b3c2-4abc-863d-194ddde959bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 19:09:27.329731 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:27.329694    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" podUID="76010446-b3c2-4abc-863d-194ddde959bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 19:09:27.816440 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:27.816374    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" podUID="76010446-b3c2-4abc-863d-194ddde959bb" containerName="main" containerID="cri-o://3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c" gracePeriod=45
Jul 27 19:09:28.079923 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.079853    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:09:28.082273 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.082250    2577 generic.go:356] "Generic (PLEG): container finished" podID="76010446-b3c2-4abc-863d-194ddde959bb" containerID="3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c" exitCode=0
Jul 27 19:09:28.082405 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.082288    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"
Jul 27 19:09:28.082405 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.082323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" event={"ID":"76010446-b3c2-4abc-863d-194ddde959bb","Type":"ContainerDied","Data":"3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c"}
Jul 27 19:09:28.082405 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.082376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc" event={"ID":"76010446-b3c2-4abc-863d-194ddde959bb","Type":"ContainerDied","Data":"d2d0df28a1ae6f0d5e90fc21e8cad811cf83dad31a378a996add915f9b03ded3"}
Jul 27 19:09:28.082591 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.082405    2577 scope.go:117] "RemoveContainer" containerID="3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c"
Jul 27 19:09:28.090640 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.090615    2577 scope.go:117] "RemoveContainer" containerID="3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c"
Jul 27 19:09:28.090891 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:09:28.090870    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c\": container with ID starting with 3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c not found: ID does not exist" containerID="3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c"
Jul 27 19:09:28.090945 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.090899    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c"} err="failed to get container status \"3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c\": rpc error: code = NotFound desc = could not find container \"3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c\": container with ID starting with 3e17192daf1bc91f837e126c5d4d5d4a7f475c619a26519cc8b3cf0f273a6b5c not found: ID does not exist"
Jul 27 19:09:28.194697 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.194665    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdmzz\" (UniqueName: \"kubernetes.io/projected/76010446-b3c2-4abc-863d-194ddde959bb-kube-api-access-gdmzz\") pod \"76010446-b3c2-4abc-863d-194ddde959bb\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") "
Jul 27 19:09:28.194867 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.194801    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76010446-b3c2-4abc-863d-194ddde959bb-tls-certs\") pod \"76010446-b3c2-4abc-863d-194ddde959bb\" (UID: \"76010446-b3c2-4abc-863d-194ddde959bb\") "
Jul 27 19:09:28.197040 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.196990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76010446-b3c2-4abc-863d-194ddde959bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "76010446-b3c2-4abc-863d-194ddde959bb" (UID: "76010446-b3c2-4abc-863d-194ddde959bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:09:28.197172 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.197044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76010446-b3c2-4abc-863d-194ddde959bb-kube-api-access-gdmzz" (OuterVolumeSpecName: "kube-api-access-gdmzz") pod "76010446-b3c2-4abc-863d-194ddde959bb" (UID: "76010446-b3c2-4abc-863d-194ddde959bb"). InnerVolumeSpecName "kube-api-access-gdmzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:09:28.296024 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.295983    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76010446-b3c2-4abc-863d-194ddde959bb-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:09:28.296024 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.296019    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdmzz\" (UniqueName: \"kubernetes.io/projected/76010446-b3c2-4abc-863d-194ddde959bb-kube-api-access-gdmzz\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:09:28.403249 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.403209    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"]
Jul 27 19:09:28.405145 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.405122    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-6dc6dff85-75kpc"]
Jul 27 19:09:28.483923 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:09:28.483885    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76010446-b3c2-4abc-863d-194ddde959bb" path="/var/lib/kubelet/pods/76010446-b3c2-4abc-863d-194ddde959bb/volumes"
Jul 27 19:10:15.208191 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:15.208146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:15.303883 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:15.303853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:21.392249 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:21.392204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:26.715046 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.715007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:26.938520 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.938487    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"]
Jul 27 19:10:26.938897 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.938881    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76010446-b3c2-4abc-863d-194ddde959bb" containerName="main"
Jul 27 19:10:26.938897 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.938899    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="76010446-b3c2-4abc-863d-194ddde959bb" containerName="main"
Jul 27 19:10:26.938981 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.938960    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="76010446-b3c2-4abc-863d-194ddde959bb" containerName="main"
Jul 27 19:10:26.941819 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.941798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:26.945480 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.945458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-epp-sa-dockercfg-cxbs6\""
Jul 27 19:10:26.945614 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.945461    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Jul 27 19:10:26.945614 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.945523    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Jul 27 19:10:26.945614 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.945461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 27 19:10:26.945614 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.945463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-4mb28\""
Jul 27 19:10:26.950087 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.949769    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"]
Jul 27 19:10:26.958030 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:26.958000    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:27.015841 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.015747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdbqs\" (UniqueName: \"kubernetes.io/projected/d22d9324-154e-41ea-a0bb-6c0d6644715e-kube-api-access-cdbqs\") pod \"rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.015841 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.015807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22d9324-154e-41ea-a0bb-6c0d6644715e-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.116345 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.116309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdbqs\" (UniqueName: \"kubernetes.io/projected/d22d9324-154e-41ea-a0bb-6c0d6644715e-kube-api-access-cdbqs\") pod \"rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.116574 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.116360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22d9324-154e-41ea-a0bb-6c0d6644715e-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.118933 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.118910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22d9324-154e-41ea-a0bb-6c0d6644715e-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.132346 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.132313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdbqs\" (UniqueName: \"kubernetes.io/projected/d22d9324-154e-41ea-a0bb-6c0d6644715e-kube-api-access-cdbqs\") pod \"rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.253556 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.253525    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:27.390920 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.390894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"]
Jul 27 19:10:27.393682 ip-10-0-135-40 kubenswrapper[2577]: W0727 19:10:27.393654    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd22d9324_154e_41ea_a0bb_6c0d6644715e.slice/crio-8bc690b8922c59f65fed040fd988d3fb83eaaf19af292988251cd15cd296ff42 WatchSource:0}: Error finding container 8bc690b8922c59f65fed040fd988d3fb83eaaf19af292988251cd15cd296ff42: Status 404 returned error can't find the container with id 8bc690b8922c59f65fed040fd988d3fb83eaaf19af292988251cd15cd296ff42
Jul 27 19:10:27.550878 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.550837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:27.633267 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:27.633175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:28.366376 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:28.366341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" event={"ID":"d22d9324-154e-41ea-a0bb-6c0d6644715e","Type":"ContainerStarted","Data":"3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9"}
Jul 27 19:10:28.366818 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:28.366402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" event={"ID":"d22d9324-154e-41ea-a0bb-6c0d6644715e","Type":"ContainerStarted","Data":"8bc690b8922c59f65fed040fd988d3fb83eaaf19af292988251cd15cd296ff42"}
Jul 27 19:10:28.366818 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:28.366494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:10:28.385756 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:28.385708    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" podStartSLOduration=2.385692584 podStartE2EDuration="2.385692584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:10:28.383626472 +0000 UTC m=+6096.462058894" watchObservedRunningTime="2026-07-27 19:10:28.385692584 +0000 UTC m=+6096.464125007"
Jul 27 19:10:44.985826 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:44.985743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:10:59.373418 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:10:59.373385    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:11:32.274807 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:32.274770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:11:32.281975 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:32.281945    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"]
Jul 27 19:11:32.374472 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:32.374424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:11:39.373506 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:39.373466    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" podUID="d22d9324-154e-41ea-a0bb-6c0d6644715e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 19:11:47.301240 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.301179    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" podUID="d22d9324-154e-41ea-a0bb-6c0d6644715e" containerName="main" containerID="cri-o://3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9" gracePeriod=45
Jul 27 19:11:47.553748 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.553682    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:11:47.619174 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.619144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22d9324-154e-41ea-a0bb-6c0d6644715e-tls-certs\") pod \"d22d9324-154e-41ea-a0bb-6c0d6644715e\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") "
Jul 27 19:11:47.619344 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.619235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdbqs\" (UniqueName: \"kubernetes.io/projected/d22d9324-154e-41ea-a0bb-6c0d6644715e-kube-api-access-cdbqs\") pod \"d22d9324-154e-41ea-a0bb-6c0d6644715e\" (UID: \"d22d9324-154e-41ea-a0bb-6c0d6644715e\") "
Jul 27 19:11:47.621359 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.621328    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d22d9324-154e-41ea-a0bb-6c0d6644715e-kube-api-access-cdbqs" (OuterVolumeSpecName: "kube-api-access-cdbqs") pod "d22d9324-154e-41ea-a0bb-6c0d6644715e" (UID: "d22d9324-154e-41ea-a0bb-6c0d6644715e"). InnerVolumeSpecName "kube-api-access-cdbqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:11:47.621359 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.621331    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d22d9324-154e-41ea-a0bb-6c0d6644715e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d22d9324-154e-41ea-a0bb-6c0d6644715e" (UID: "d22d9324-154e-41ea-a0bb-6c0d6644715e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:11:47.720410 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.720372    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdbqs\" (UniqueName: \"kubernetes.io/projected/d22d9324-154e-41ea-a0bb-6c0d6644715e-kube-api-access-cdbqs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:11:47.720410 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.720404    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d22d9324-154e-41ea-a0bb-6c0d6644715e-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:11:47.745646 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.745614    2577 generic.go:356] "Generic (PLEG): container finished" podID="d22d9324-154e-41ea-a0bb-6c0d6644715e" containerID="3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9" exitCode=0
Jul 27 19:11:47.745794 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.745662    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"
Jul 27 19:11:47.745794 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.745701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" event={"ID":"d22d9324-154e-41ea-a0bb-6c0d6644715e","Type":"ContainerDied","Data":"3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9"}
Jul 27 19:11:47.745794 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.745750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h" event={"ID":"d22d9324-154e-41ea-a0bb-6c0d6644715e","Type":"ContainerDied","Data":"8bc690b8922c59f65fed040fd988d3fb83eaaf19af292988251cd15cd296ff42"}
Jul 27 19:11:47.745794 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.745773    2577 scope.go:117] "RemoveContainer" containerID="3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9"
Jul 27 19:11:47.755494 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.755470    2577 scope.go:117] "RemoveContainer" containerID="3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9"
Jul 27 19:11:47.755752 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:11:47.755735    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9\": container with ID starting with 3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9 not found: ID does not exist" containerID="3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9"
Jul 27 19:11:47.755807 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.755760    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9"} err="failed to get container status \"3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9\": rpc error: code = NotFound desc = could not find container \"3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9\": container with ID starting with 3786959db341c15914dcc2406922444330ce583626b65337251a521efeb584a9 not found: ID does not exist"
Jul 27 19:11:47.767710 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.767689    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"]
Jul 27 19:11:47.771199 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:47.771176    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-59d79f7cfc-2z96h"]
Jul 27 19:11:48.489589 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:48.489555    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d22d9324-154e-41ea-a0bb-6c0d6644715e" path="/var/lib/kubelet/pods/d22d9324-154e-41ea-a0bb-6c0d6644715e/volumes"
Jul 27 19:11:50.770631 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:50.770579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:11:50.874166 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:50.874131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:11:50.971125 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:50.971088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:11:57.030725 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:11:57.030689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:12:02.714718 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:02.714676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:12:02.777800 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:02.777765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:12:29.080850 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:29.080768    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-7584bddd84-vvhxz"]
Jul 27 19:12:29.081384 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:29.080976    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz" podUID="61b5d9bd-868f-48f1-9a82-0729a06be2dd" containerName="manager" containerID="cri-o://f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18" gracePeriod=30
Jul 27 19:12:40.838374 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.838348    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 19:12:40.882650 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.882620    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t796c\" (UniqueName: \"kubernetes.io/projected/61b5d9bd-868f-48f1-9a82-0729a06be2dd-kube-api-access-t796c\") pod \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") "
Jul 27 19:12:40.882809 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.882755    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert\") pod \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\" (UID: \"61b5d9bd-868f-48f1-9a82-0729a06be2dd\") "
Jul 27 19:12:40.884869 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.884833    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b5d9bd-868f-48f1-9a82-0729a06be2dd-kube-api-access-t796c" (OuterVolumeSpecName: "kube-api-access-t796c") pod "61b5d9bd-868f-48f1-9a82-0729a06be2dd" (UID: "61b5d9bd-868f-48f1-9a82-0729a06be2dd"). InnerVolumeSpecName "kube-api-access-t796c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:12:40.884869 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.884846    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert" (OuterVolumeSpecName: "cert") pod "61b5d9bd-868f-48f1-9a82-0729a06be2dd" (UID: "61b5d9bd-868f-48f1-9a82-0729a06be2dd"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:12:40.984316 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.984280    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t796c\" (UniqueName: \"kubernetes.io/projected/61b5d9bd-868f-48f1-9a82-0729a06be2dd-kube-api-access-t796c\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:12:40.984316 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.984310    2577 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61b5d9bd-868f-48f1-9a82-0729a06be2dd-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:12:40.999758 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.999730    2577 generic.go:356] "Generic (PLEG): container finished" podID="61b5d9bd-868f-48f1-9a82-0729a06be2dd" containerID="f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18" exitCode=0
Jul 27 19:12:40.999920 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.999772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz" event={"ID":"61b5d9bd-868f-48f1-9a82-0729a06be2dd","Type":"ContainerDied","Data":"f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18"}
Jul 27 19:12:40.999920 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.999781    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz"
Jul 27 19:12:40.999920 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.999837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-7584bddd84-vvhxz" event={"ID":"61b5d9bd-868f-48f1-9a82-0729a06be2dd","Type":"ContainerDied","Data":"6de988ad22c7e2d80f085847fbd4361fc1e8b4bec05fddab4820dcf4a209416f"}
Jul 27 19:12:40.999920 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:40.999858    2577 scope.go:117] "RemoveContainer" containerID="f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18"
Jul 27 19:12:41.013931 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:41.013912    2577 scope.go:117] "RemoveContainer" containerID="f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18"
Jul 27 19:12:41.014176 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:12:41.014159    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18\": container with ID starting with f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18 not found: ID does not exist" containerID="f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18"
Jul 27 19:12:41.014227 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:41.014183    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18"} err="failed to get container status \"f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18\": rpc error: code = NotFound desc = could not find container \"f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18\": container with ID starting with f631e8ef4c8d7dbf4556222eda14887f3b8ed85527be7a94320483f310213c18 not found: ID does not exist"
Jul 27 19:12:41.022000 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:41.021980    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-7584bddd84-vvhxz"]
Jul 27 19:12:41.024950 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:41.024930    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/llmisvc-controller-manager-7584bddd84-vvhxz"]
Jul 27 19:12:42.483841 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:42.483810    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61b5d9bd-868f-48f1-9a82-0729a06be2dd" path="/var/lib/kubelet/pods/61b5d9bd-868f-48f1-9a82-0729a06be2dd/volumes"
Jul 27 19:12:47.812534 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:12:47.812499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:21.690824 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:21.690785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:27.657877 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:27.657839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:27.758422 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:27.758384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:29.931379 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:29.931336    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"]
Jul 27 19:13:29.933320 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:29.933292    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"]
Jul 27 19:13:29.939523 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:29.939493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:29.943074 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:13:29.943040    2577 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 27 19:13:29.943074 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 19:13:29.943074 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 19:13:29.943074 ip-10-0-135-40 kubenswrapper[2577]:  > containerID="8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f" cmd=["/bin/sleep","15"]
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:29.943098    2577 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" message="" err=<
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]:  >
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:13:29.943114    2577 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]:         , stdout: , stderr: , exit code -1
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]:  > pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" podUID="591c360d-cd22-4518-adbc-674139848e5a" containerName="main" containerID="cri-o://8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f"
Jul 27 19:13:29.943325 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:29.943156    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" podUID="591c360d-cd22-4518-adbc-674139848e5a" containerName="main" containerID="cri-o://8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f" gracePeriod=60
Jul 27 19:13:30.000202 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.000167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:30.099708 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.099663    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" podUID="591c360d-cd22-4518-adbc-674139848e5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": dial tcp 10.134.0.55:8000: connect: connection refused"
Jul 27 19:13:30.234630 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.234593    2577 generic.go:356] "Generic (PLEG): container finished" podID="591c360d-cd22-4518-adbc-674139848e5a" containerID="8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f" exitCode=0
Jul 27 19:13:30.234805 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.234664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" event={"ID":"591c360d-cd22-4518-adbc-674139848e5a","Type":"ContainerDied","Data":"8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f"}
Jul 27 19:13:30.696260 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.696231    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 19:13:30.714349 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714055    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-model-cache\") pod \"591c360d-cd22-4518-adbc-674139848e5a\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") "
Jul 27 19:13:30.714554 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714458    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/591c360d-cd22-4518-adbc-674139848e5a-tls-certs\") pod \"591c360d-cd22-4518-adbc-674139848e5a\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") "
Jul 27 19:13:30.714554 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714513    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-dshm\") pod \"591c360d-cd22-4518-adbc-674139848e5a\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") "
Jul 27 19:13:30.714554 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-home\") pod \"591c360d-cd22-4518-adbc-674139848e5a\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") "
Jul 27 19:13:30.714780 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714591    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8snr\" (UniqueName: \"kubernetes.io/projected/591c360d-cd22-4518-adbc-674139848e5a-kube-api-access-n8snr\") pod \"591c360d-cd22-4518-adbc-674139848e5a\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") "
Jul 27 19:13:30.714780 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-tmp-dir\") pod \"591c360d-cd22-4518-adbc-674139848e5a\" (UID: \"591c360d-cd22-4518-adbc-674139848e5a\") "
Jul 27 19:13:30.714780 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714707    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-model-cache" (OuterVolumeSpecName: "model-cache") pod "591c360d-cd22-4518-adbc-674139848e5a" (UID: "591c360d-cd22-4518-adbc-674139848e5a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:30.714968 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.714874    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:30.715048 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.715027    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-home" (OuterVolumeSpecName: "home") pod "591c360d-cd22-4518-adbc-674139848e5a" (UID: "591c360d-cd22-4518-adbc-674139848e5a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:30.715363 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.715309    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "591c360d-cd22-4518-adbc-674139848e5a" (UID: "591c360d-cd22-4518-adbc-674139848e5a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:30.717125 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.717103    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/591c360d-cd22-4518-adbc-674139848e5a-kube-api-access-n8snr" (OuterVolumeSpecName: "kube-api-access-n8snr") pod "591c360d-cd22-4518-adbc-674139848e5a" (UID: "591c360d-cd22-4518-adbc-674139848e5a"). InnerVolumeSpecName "kube-api-access-n8snr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:13:30.717252 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.717216    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-dshm" (OuterVolumeSpecName: "dshm") pod "591c360d-cd22-4518-adbc-674139848e5a" (UID: "591c360d-cd22-4518-adbc-674139848e5a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:13:30.717309 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.717293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/591c360d-cd22-4518-adbc-674139848e5a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "591c360d-cd22-4518-adbc-674139848e5a" (UID: "591c360d-cd22-4518-adbc-674139848e5a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:13:30.815781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.815690    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:30.815781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.815722    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:30.815781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.815731    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8snr\" (UniqueName: \"kubernetes.io/projected/591c360d-cd22-4518-adbc-674139848e5a-kube-api-access-n8snr\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:30.815781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.815740    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/591c360d-cd22-4518-adbc-674139848e5a-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:30.815781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:30.815749    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/591c360d-cd22-4518-adbc-674139848e5a-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:31.240417 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:31.240389    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"
Jul 27 19:13:31.240860 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:31.240388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl" event={"ID":"591c360d-cd22-4518-adbc-674139848e5a","Type":"ContainerDied","Data":"12ff9ac424efa64dd05742ffe7173c6dbab1be2313a99576315421323778f00d"}
Jul 27 19:13:31.240860 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:31.240525    2577 scope.go:117] "RemoveContainer" containerID="8c6d1ffb0e848b55f8c6d80cf6c78d792c2954901a38d6e9b6fe68ff5b50654f"
Jul 27 19:13:31.260772 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:31.260736    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"]
Jul 27 19:13:31.264608 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:31.264583    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-wnpkl"]
Jul 27 19:13:32.482920 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:32.482886    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="591c360d-cd22-4518-adbc-674139848e5a" path="/var/lib/kubelet/pods/591c360d-cd22-4518-adbc-674139848e5a/volumes"
Jul 27 19:13:36.067741 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:36.067703    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" podUID="17f13d3d-54b3-4d96-acde-f36c3da56da2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 27 19:13:44.955414 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:44.955303    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" podUID="17f13d3d-54b3-4d96-acde-f36c3da56da2" containerName="main" containerID="cri-o://faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894" gracePeriod=45
Jul 27 19:13:45.199816 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.199790    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 19:13:45.237763 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.237728    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f13d3d-54b3-4d96-acde-f36c3da56da2-tls-certs\") pod \"17f13d3d-54b3-4d96-acde-f36c3da56da2\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") "
Jul 27 19:13:45.237964 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.237778    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2rrv\" (UniqueName: \"kubernetes.io/projected/17f13d3d-54b3-4d96-acde-f36c3da56da2-kube-api-access-b2rrv\") pod \"17f13d3d-54b3-4d96-acde-f36c3da56da2\" (UID: \"17f13d3d-54b3-4d96-acde-f36c3da56da2\") "
Jul 27 19:13:45.239863 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.239830    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17f13d3d-54b3-4d96-acde-f36c3da56da2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "17f13d3d-54b3-4d96-acde-f36c3da56da2" (UID: "17f13d3d-54b3-4d96-acde-f36c3da56da2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:13:45.240009 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.239900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f13d3d-54b3-4d96-acde-f36c3da56da2-kube-api-access-b2rrv" (OuterVolumeSpecName: "kube-api-access-b2rrv") pod "17f13d3d-54b3-4d96-acde-f36c3da56da2" (UID: "17f13d3d-54b3-4d96-acde-f36c3da56da2"). InnerVolumeSpecName "kube-api-access-b2rrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:13:45.305672 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.305641    2577 generic.go:356] "Generic (PLEG): container finished" podID="17f13d3d-54b3-4d96-acde-f36c3da56da2" containerID="faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894" exitCode=0
Jul 27 19:13:45.305804 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.305688    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"
Jul 27 19:13:45.305804 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.305730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" event={"ID":"17f13d3d-54b3-4d96-acde-f36c3da56da2","Type":"ContainerDied","Data":"faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894"}
Jul 27 19:13:45.305804 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.305777    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b" event={"ID":"17f13d3d-54b3-4d96-acde-f36c3da56da2","Type":"ContainerDied","Data":"e2eaaa75694ca9e44dcd3efe5ffb04a7583e92f830f720d1d315dee0e01156e8"}
Jul 27 19:13:45.305804 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.305799    2577 scope.go:117] "RemoveContainer" containerID="faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894"
Jul 27 19:13:45.314917 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.314896    2577 scope.go:117] "RemoveContainer" containerID="faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894"
Jul 27 19:13:45.315191 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:13:45.315171    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894\": container with ID starting with faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894 not found: ID does not exist" containerID="faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894"
Jul 27 19:13:45.315239 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.315200    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894"} err="failed to get container status \"faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894\": rpc error: code = NotFound desc = could not find container \"faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894\": container with ID starting with faa65c3762adfe50d7fe8b4bb8ce30f14776bcbdc7b41e0a3fd82d1a951fc894 not found: ID does not exist"
Jul 27 19:13:45.327250 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.327219    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"]
Jul 27 19:13:45.331074 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.331051    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-router-scheduler-6694c4458b-rc29b"]
Jul 27 19:13:45.338783 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.338769    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f13d3d-54b3-4d96-acde-f36c3da56da2-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:45.338845 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:45.338787    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2rrv\" (UniqueName: \"kubernetes.io/projected/17f13d3d-54b3-4d96-acde-f36c3da56da2-kube-api-access-b2rrv\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:13:46.484847 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:46.484806    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17f13d3d-54b3-4d96-acde-f36c3da56da2" path="/var/lib/kubelet/pods/17f13d3d-54b3-4d96-acde-f36c3da56da2/volumes"
Jul 27 19:13:51.970042 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sth4s/must-gather-k76f6"]
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970340    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d22d9324-154e-41ea-a0bb-6c0d6644715e" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970350    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d22d9324-154e-41ea-a0bb-6c0d6644715e" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970358    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61b5d9bd-868f-48f1-9a82-0729a06be2dd" containerName="manager"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970363    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="61b5d9bd-868f-48f1-9a82-0729a06be2dd" containerName="manager"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970371    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="591c360d-cd22-4518-adbc-674139848e5a" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970378    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="591c360d-cd22-4518-adbc-674139848e5a" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970385    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f13d3d-54b3-4d96-acde-f36c3da56da2" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f13d3d-54b3-4d96-acde-f36c3da56da2" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970468    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="61b5d9bd-868f-48f1-9a82-0729a06be2dd" containerName="manager"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970480    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d22d9324-154e-41ea-a0bb-6c0d6644715e" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970490    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="591c360d-cd22-4518-adbc-674139848e5a" containerName="main"
Jul 27 19:13:51.970491 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.970499    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="17f13d3d-54b3-4d96-acde-f36c3da56da2" containerName="main"
Jul 27 19:13:51.974880 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.974860    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:51.977242 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.977214    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sth4s\"/\"kube-root-ca.crt\""
Jul 27 19:13:51.977374 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.977264    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sth4s\"/\"openshift-service-ca.crt\""
Jul 27 19:13:51.982918 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.982887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sth4s/must-gather-k76f6"]
Jul 27 19:13:51.989143 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:51.989107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:52.095425 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.095390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/547e1354-f12b-4d0a-8ada-82e52c95164a-must-gather-output\") pod \"must-gather-k76f6\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") " pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.095425 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.095453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjd2s\" (UniqueName: \"kubernetes.io/projected/547e1354-f12b-4d0a-8ada-82e52c95164a-kube-api-access-sjd2s\") pod \"must-gather-k76f6\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") " pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.196719 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.196679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/547e1354-f12b-4d0a-8ada-82e52c95164a-must-gather-output\") pod \"must-gather-k76f6\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") " pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.196719 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.196726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjd2s\" (UniqueName: \"kubernetes.io/projected/547e1354-f12b-4d0a-8ada-82e52c95164a-kube-api-access-sjd2s\") pod \"must-gather-k76f6\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") " pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.197111 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.197091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/547e1354-f12b-4d0a-8ada-82e52c95164a-must-gather-output\") pod \"must-gather-k76f6\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") " pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.204592 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.204562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjd2s\" (UniqueName: \"kubernetes.io/projected/547e1354-f12b-4d0a-8ada-82e52c95164a-kube-api-access-sjd2s\") pod \"must-gather-k76f6\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") " pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.293148 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.293056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:13:52.426309 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.426287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sth4s/must-gather-k76f6"]
Jul 27 19:13:52.428838 ip-10-0-135-40 kubenswrapper[2577]: W0727 19:13:52.428809    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod547e1354_f12b_4d0a_8ada_82e52c95164a.slice/crio-28e4d9e1cd18bdb94247fe274228b4cc0b403e100ab6566198d38892ba1121e3 WatchSource:0}: Error finding container 28e4d9e1cd18bdb94247fe274228b4cc0b403e100ab6566198d38892ba1121e3: Status 404 returned error can't find the container with id 28e4d9e1cd18bdb94247fe274228b4cc0b403e100ab6566198d38892ba1121e3
Jul 27 19:13:52.430582 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.430562    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:13:52.861454 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.861335    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:13:52.879040 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.862194    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:13:52.879040 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.867017    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 19:13:52.881176 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.881157    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:13:52.882269 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.882248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:13:52.887781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:52.887764    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nld5j_c5360283-57fc-49d4-87bf-9322bf5e5665/ovn-acl-logging/0.log"
Jul 27 19:13:53.358717 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:53.358671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sth4s/must-gather-k76f6" event={"ID":"547e1354-f12b-4d0a-8ada-82e52c95164a","Type":"ContainerStarted","Data":"28e4d9e1cd18bdb94247fe274228b4cc0b403e100ab6566198d38892ba1121e3"}
Jul 27 19:13:53.620292 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:53.620218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:13:59.395160 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:59.395127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sth4s/must-gather-k76f6" event={"ID":"547e1354-f12b-4d0a-8ada-82e52c95164a","Type":"ContainerStarted","Data":"cd057ceecce101e5dec79f5a212bda6b1db12719fa9f2d7bed249808f45f58d1"}
Jul 27 19:13:59.395611 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:59.395169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sth4s/must-gather-k76f6" event={"ID":"547e1354-f12b-4d0a-8ada-82e52c95164a","Type":"ContainerStarted","Data":"811afdca28522d0c084dfdbb6d62e20e0b0423ecf2498d852e30f6607456dc0f"}
Jul 27 19:13:59.410685 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:13:59.410614    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sth4s/must-gather-k76f6" podStartSLOduration=2.307115617 podStartE2EDuration="8.410593536s" totalImagesPullingTime="6.103477919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:13:59.410276814 +0000 UTC m=+6307.488709238" watchObservedRunningTime="2026-07-27 19:13:59.410593536 +0000 UTC m=+6307.489025960"
Jul 27 19:14:14.086882 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:14.086838    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:14:14.152401 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:14.152373    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:14:15.166295 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:15.166259    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:14:15.220678 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:15.220647    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:14:16.260316 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:16.260284    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:14:16.324989 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:16.324954    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:14:17.364110 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:17.364079    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:14:17.559214 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:17.559187    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:14:18.542621 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:18.542589    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:18.837182 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:18.837099    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:18.853285 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:18.853257    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:19.896636 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:19.896595    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:20.198944 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:20.198866    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:20.218341 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:20.218311    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:21.214817 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:21.214787    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:21.506450 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:21.506344    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:21.527120 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:21.527093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:22.507022 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:22.506989    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:22.792570 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:22.792499    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:22.810360 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:22.810329    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:23.797475 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:23.797442    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:24.089286 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:24.089202    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:24.108979 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:24.108951    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:25.095580 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:25.095543    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:25.399986 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:25.399912    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:25.415293 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:25.415261    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:26.407320 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:26.407285    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:26.704588 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:26.704514    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:26.722162 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:26.722132    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:27.702182 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:27.702152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:27.976531 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:27.976488    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:27.992231 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:27.992204    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:28.967935 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:28.967905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:29.253820 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:29.253725    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:29.273148 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:29.273121    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:30.281823 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:30.281787    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:30.573171 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:30.573093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:30.589398 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:30.589369    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:31.576652 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:31.576626    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:31.862106 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:31.862035    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:31.877617 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:31.877587    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:32.857351 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:32.857315    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:33.156589 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:33.156513    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:33.175269 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:33.175229    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:34.163770 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:34.163731    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:34.465149 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:34.465122    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:34.483214 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:34.483185    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:35.587785 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:35.587756    2577 generic.go:356] "Generic (PLEG): container finished" podID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerID="811afdca28522d0c084dfdbb6d62e20e0b0423ecf2498d852e30f6607456dc0f" exitCode=0
Jul 27 19:14:35.588253 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:35.587831    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sth4s/must-gather-k76f6" event={"ID":"547e1354-f12b-4d0a-8ada-82e52c95164a","Type":"ContainerDied","Data":"811afdca28522d0c084dfdbb6d62e20e0b0423ecf2498d852e30f6607456dc0f"}
Jul 27 19:14:35.588253 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:35.588149    2577 scope.go:117] "RemoveContainer" containerID="811afdca28522d0c084dfdbb6d62e20e0b0423ecf2498d852e30f6607456dc0f"
Jul 27 19:14:35.987276 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:35.987240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-sth4s_must-gather-k76f6_547e1354-f12b-4d0a-8ada-82e52c95164a/gather/0.log"
Jul 27 19:14:36.441134 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.441042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:14:36.590977 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.590944    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r8n2x/must-gather-nk8b4"]
Jul 27 19:14:36.595767 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.595748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.598226 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.598204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r8n2x\"/\"kube-root-ca.crt\""
Jul 27 19:14:36.598332 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.598207    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r8n2x\"/\"openshift-service-ca.crt\""
Jul 27 19:14:36.599326 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.599306    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-r8n2x\"/\"default-dockercfg-xsd9b\""
Jul 27 19:14:36.601608 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.601589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r8n2x/must-gather-nk8b4"]
Jul 27 19:14:36.602062 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.602034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:14:36.695095 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.695024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c91bb32e-3e45-4de8-af03-fec6ca8f1dcc-must-gather-output\") pod \"must-gather-nk8b4\" (UID: \"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc\") " pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.695095 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.695092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6647\" (UniqueName: \"kubernetes.io/projected/c91bb32e-3e45-4de8-af03-fec6ca8f1dcc-kube-api-access-d6647\") pod \"must-gather-nk8b4\" (UID: \"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc\") " pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.701791 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.701761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:14:36.796100 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.796064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6647\" (UniqueName: \"kubernetes.io/projected/c91bb32e-3e45-4de8-af03-fec6ca8f1dcc-kube-api-access-d6647\") pod \"must-gather-nk8b4\" (UID: \"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc\") " pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.796296 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.796145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c91bb32e-3e45-4de8-af03-fec6ca8f1dcc-must-gather-output\") pod \"must-gather-nk8b4\" (UID: \"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc\") " pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.796479 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.796461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c91bb32e-3e45-4de8-af03-fec6ca8f1dcc-must-gather-output\") pod \"must-gather-nk8b4\" (UID: \"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc\") " pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.809614 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.809592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6647\" (UniqueName: \"kubernetes.io/projected/c91bb32e-3e45-4de8-af03-fec6ca8f1dcc-kube-api-access-d6647\") pod \"must-gather-nk8b4\" (UID: \"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc\") " pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:36.906089 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:36.906062    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r8n2x/must-gather-nk8b4"
Jul 27 19:14:37.040792 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:37.040762    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r8n2x/must-gather-nk8b4"]
Jul 27 19:14:37.043458 ip-10-0-135-40 kubenswrapper[2577]: W0727 19:14:37.043418    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc91bb32e_3e45_4de8_af03_fec6ca8f1dcc.slice/crio-20125bc673e38e0ea5e7c026caeeb60c26bbffeb39448c9795de0e017512c700 WatchSource:0}: Error finding container 20125bc673e38e0ea5e7c026caeeb60c26bbffeb39448c9795de0e017512c700: Status 404 returned error can't find the container with id 20125bc673e38e0ea5e7c026caeeb60c26bbffeb39448c9795de0e017512c700
Jul 27 19:14:37.599771 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:37.599738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r8n2x/must-gather-nk8b4" event={"ID":"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc","Type":"ContainerStarted","Data":"20125bc673e38e0ea5e7c026caeeb60c26bbffeb39448c9795de0e017512c700"}
Jul 27 19:14:38.608871 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:38.608600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r8n2x/must-gather-nk8b4" event={"ID":"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc","Type":"ContainerStarted","Data":"5bdbcb486e84f8e8149667fb1cf3188c1d18134a9a2563d766fbbe95190423e4"}
Jul 27 19:14:38.608871 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:38.608653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r8n2x/must-gather-nk8b4" event={"ID":"c91bb32e-3e45-4de8-af03-fec6ca8f1dcc","Type":"ContainerStarted","Data":"e071cc08a7af6870858963e82037200ec21446c5d30d75536fd25d8378c9a3e6"}
Jul 27 19:14:38.627336 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:38.627252    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-r8n2x/must-gather-nk8b4" podStartSLOduration=1.956581303 podStartE2EDuration="2.627230052s" totalImagesPullingTime="670.648749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:14:38.624072616 +0000 UTC m=+6346.702505054" watchObservedRunningTime="2026-07-27 19:14:38.627230052 +0000 UTC m=+6346.705662475"
Jul 27 19:14:39.216074 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:39.216031    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qxkrc_c9cb2437-7ae9-44d4-8f5f-927e38a41769/global-pull-secret-syncer/0.log"
Jul 27 19:14:39.348594 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:39.348556    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log"
Jul 27 19:14:39.569620 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:39.569542    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-40.ec2.internal_55803ecb689b631e773ecebce7207b60/haproxy/0.log"
Jul 27 19:14:41.437755 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.437712    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-sth4s/must-gather-k76f6"]
Jul 27 19:14:41.438254 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.437925    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-sth4s/must-gather-k76f6" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="copy" containerID="cri-o://cd057ceecce101e5dec79f5a212bda6b1db12719fa9f2d7bed249808f45f58d1" gracePeriod=2
Jul 27 19:14:41.440912 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:14:41.440869    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-k76f6\" is forbidden: User \"system:node:ip-10-0-135-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-sth4s\": no relationship found between node 'ip-10-0-135-40.ec2.internal' and this object" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:14:41.443336 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.443318    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-sth4s/must-gather-k76f6"]
Jul 27 19:14:41.629981 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.629934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-sth4s_must-gather-k76f6_547e1354-f12b-4d0a-8ada-82e52c95164a/copy/0.log"
Jul 27 19:14:41.630411 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.630377    2577 generic.go:356] "Generic (PLEG): container finished" podID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerID="cd057ceecce101e5dec79f5a212bda6b1db12719fa9f2d7bed249808f45f58d1" exitCode=143
Jul 27 19:14:41.829384 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.829344    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-sth4s_must-gather-k76f6_547e1354-f12b-4d0a-8ada-82e52c95164a/copy/0.log"
Jul 27 19:14:41.837695 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.837644    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:14:41.842682 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:14:41.842621    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-k76f6\" is forbidden: User \"system:node:ip-10-0-135-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-sth4s\": no relationship found between node 'ip-10-0-135-40.ec2.internal' and this object" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:14:41.946842 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.946795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/547e1354-f12b-4d0a-8ada-82e52c95164a-must-gather-output\") pod \"547e1354-f12b-4d0a-8ada-82e52c95164a\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") "
Jul 27 19:14:41.946842 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.946846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjd2s\" (UniqueName: \"kubernetes.io/projected/547e1354-f12b-4d0a-8ada-82e52c95164a-kube-api-access-sjd2s\") pod \"547e1354-f12b-4d0a-8ada-82e52c95164a\" (UID: \"547e1354-f12b-4d0a-8ada-82e52c95164a\") "
Jul 27 19:14:41.956931 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.953409    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547e1354-f12b-4d0a-8ada-82e52c95164a-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "547e1354-f12b-4d0a-8ada-82e52c95164a" (UID: "547e1354-f12b-4d0a-8ada-82e52c95164a"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:14:41.964621 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:41.964574    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547e1354-f12b-4d0a-8ada-82e52c95164a-kube-api-access-sjd2s" (OuterVolumeSpecName: "kube-api-access-sjd2s") pod "547e1354-f12b-4d0a-8ada-82e52c95164a" (UID: "547e1354-f12b-4d0a-8ada-82e52c95164a"). InnerVolumeSpecName "kube-api-access-sjd2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:14:42.048025 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.047928    2577 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/547e1354-f12b-4d0a-8ada-82e52c95164a-must-gather-output\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:14:42.048025 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.047975    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjd2s\" (UniqueName: \"kubernetes.io/projected/547e1354-f12b-4d0a-8ada-82e52c95164a-kube-api-access-sjd2s\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\""
Jul 27 19:14:42.484838 ip-10-0-135-40 kubenswrapper[2577]: E0727 19:14:42.484775    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-k76f6\" is forbidden: User \"system:node:ip-10-0-135-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-sth4s\": no relationship found between node 'ip-10-0-135-40.ec2.internal' and this object" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:14:42.491645 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.489344    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" path="/var/lib/kubelet/pods/547e1354-f12b-4d0a-8ada-82e52c95164a/volumes"
Jul 27 19:14:42.639743 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.639710    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-sth4s_must-gather-k76f6_547e1354-f12b-4d0a-8ada-82e52c95164a/copy/0.log"
Jul 27 19:14:42.640236 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.640215    2577 scope.go:117] "RemoveContainer" containerID="cd057ceecce101e5dec79f5a212bda6b1db12719fa9f2d7bed249808f45f58d1"
Jul 27 19:14:42.640415 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.640386    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sth4s/must-gather-k76f6"
Jul 27 19:14:42.664444 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.660660    2577 scope.go:117] "RemoveContainer" containerID="811afdca28522d0c084dfdbb6d62e20e0b0423ecf2498d852e30f6607456dc0f"
Jul 27 19:14:42.846488 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.846367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:14:42.911771 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:42.911731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lb2p_8bc14a53-bf9c-4c74-8b11-2953f0979658/konnectivity-agent/0.log" limit=1024
Jul 27 19:14:44.581692 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:44.581661    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-56fb6f6856-t4rz2_781acdec-7b07-43ff-9905-46d7bec79e89/manager/0.log"
Jul 27 19:14:44.930700 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:44.930614    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-fcvd7_b7ab4268-cbe2-4f53-b5c2-46fe7d7985a3/limitador/0.log"
Jul 27 19:14:44.959074 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:44.959040    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-774d94476f-4cd9s_d26d5fe0-b579-4098-bd73-2c2d76407a20/manager/0.log"
Jul 27 19:14:46.458184 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.458148    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/1.log"
Jul 27 19:14:46.551648 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.551619    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-4pw8b_024b9af0-76c5-41b8-87d7-a763b6f1a919/cluster-monitoring-operator/0.log"
Jul 27 19:14:46.660498 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.660396    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f77db5df6-9w52q_a8fc1c9e-8278-4c60-86a5-feeefdbd22a5/metrics-server/0.log"
Jul 27 19:14:46.688974 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.688939    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-bcrmc_7dd69eae-b688-4555-86b4-a22777055e8e/monitoring-plugin/0.log"
Jul 27 19:14:46.843667 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.843631    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kcv6b_2f45f9ff-9b97-4b95-8834-80308aadc3a1/node-exporter/0.log"
Jul 27 19:14:46.866136 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.866067    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kcv6b_2f45f9ff-9b97-4b95-8834-80308aadc3a1/kube-rbac-proxy/0.log"
Jul 27 19:14:46.889337 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:46.889307    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kcv6b_2f45f9ff-9b97-4b95-8834-80308aadc3a1/init-textfile/0.log"
Jul 27 19:14:47.311161 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:47.311132    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-jcnth_a07f34b7-b180-4541-af7a-84217e5276b2/prometheus-operator-admission-webhook/0.log"
Jul 27 19:14:47.338635 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:47.338595    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d5dd789bc-gkd27_a6890efe-917b-47ef-af71-c60c6e895c04/telemeter-client/0.log"
Jul 27 19:14:47.359873 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:47.359800    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d5dd789bc-gkd27_a6890efe-917b-47ef-af71-c60c6e895c04/reload/0.log"
Jul 27 19:14:47.380632 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:47.380600    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d5dd789bc-gkd27_a6890efe-917b-47ef-af71-c60c6e895c04/kube-rbac-proxy/0.log"
Jul 27 19:14:49.081423 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081392    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"]
Jul 27 19:14:49.082028 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081764    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="copy"
Jul 27 19:14:49.082028 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081781    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="copy"
Jul 27 19:14:49.082028 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081807    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="gather"
Jul 27 19:14:49.082028 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081816    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="gather"
Jul 27 19:14:49.082028 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081930    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="gather"
Jul 27 19:14:49.082028 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.081944    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="547e1354-f12b-4d0a-8ada-82e52c95164a" containerName="copy"
Jul 27 19:14:49.086090 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.086064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.087353 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.087290    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/1.log"
Jul 27 19:14:49.095716 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.095695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-pwdzm_79d2f48c-0254-434b-a3bf-455dd45f270a/console-operator/2.log"
Jul 27 19:14:49.098678 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.098562    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"]
Jul 27 19:14:49.231673 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.231633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-lib-modules\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.231841 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.231682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-podres\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.231841 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.231809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpldx\" (UniqueName: \"kubernetes.io/projected/c1def41f-a183-4d78-814b-501832f28e34-kube-api-access-mpldx\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.231925 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.231843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-sys\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.231925 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.231881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-proc\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333233 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-lib-modules\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333233 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-podres\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333498 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpldx\" (UniqueName: \"kubernetes.io/projected/c1def41f-a183-4d78-814b-501832f28e34-kube-api-access-mpldx\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333498 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-sys\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333498 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-proc\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333498 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-lib-modules\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333498 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-sys\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-proc\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.333781 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.333542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c1def41f-a183-4d78-814b-501832f28e34-podres\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.343017 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.342979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpldx\" (UniqueName: \"kubernetes.io/projected/c1def41f-a183-4d78-814b-501832f28e34-kube-api-access-mpldx\") pod \"perf-node-gather-daemonset-zxlrz\" (UID: \"c1def41f-a183-4d78-814b-501832f28e34\") " pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.410136 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.410093    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.549454 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.548159    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-68d5c4bcf9-b6sdd_f24341b9-4f61-4379-b851-243a01798834/console/0.log"
Jul 27 19:14:49.584966 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.584941    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"]
Jul 27 19:14:49.587231 ip-10-0-135-40 kubenswrapper[2577]: W0727 19:14:49.587201    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc1def41f_a183_4d78_814b_501832f28e34.slice/crio-a9ca4cb126276f5ab23cfc0642f4bbb8fd9045384a96a53c37eb100f389e2608 WatchSource:0}: Error finding container a9ca4cb126276f5ab23cfc0642f4bbb8fd9045384a96a53c37eb100f389e2608: Status 404 returned error can't find the container with id a9ca4cb126276f5ab23cfc0642f4bbb8fd9045384a96a53c37eb100f389e2608
Jul 27 19:14:49.702469 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.701700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz" event={"ID":"c1def41f-a183-4d78-814b-501832f28e34","Type":"ContainerStarted","Data":"0698171e37b00758015ed5d18d77110081eecddf0fe491fa55ade6dbbe4b637d"}
Jul 27 19:14:49.702469 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.701787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz" event={"ID":"c1def41f-a183-4d78-814b-501832f28e34","Type":"ContainerStarted","Data":"a9ca4cb126276f5ab23cfc0642f4bbb8fd9045384a96a53c37eb100f389e2608"}
Jul 27 19:14:49.702469 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.701980    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:49.719051 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:49.719000    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz" podStartSLOduration=0.718986188 podStartE2EDuration="718.986188ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:14:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:14:49.717904597 +0000 UTC m=+6357.796337037" watchObservedRunningTime="2026-07-27 19:14:49.718986188 +0000 UTC m=+6357.797418610"
Jul 27 19:14:50.113807 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:50.113782    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-mdrqd_3c1da1a1-2b28-45bf-9cac-5ed5c01b8cc4/volume-data-source-validator/0.log"
Jul 27 19:14:51.064062 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:51.064032    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t4lwh_42cb6f4f-5199-41c8-8f79-4c847ede7bcd/dns/0.log"
Jul 27 19:14:51.089510 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:51.089475    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t4lwh_42cb6f4f-5199-41c8-8f79-4c847ede7bcd/kube-rbac-proxy/0.log"
Jul 27 19:14:51.195580 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:51.195538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w84kh_1cfaef06-f66e-4e0d-9cae-01a4f177dd39/dns-node-resolver/0.log"
Jul 27 19:14:51.714210 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:51.714179    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d8bcdfb7f-gtf7k_c4d74892-4cd7-451c-9b51-6d96d48f3fa7/registry/0.log"
Jul 27 19:14:51.741965 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:51.741938    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-72mk7_75d73dec-5e9b-48a8-882b-4f67668fb911/node-ca/0.log"
Jul 27 19:14:53.069692 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:53.069653    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:14:53.139570 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:53.139539    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:14:53.615268 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:53.615240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dmr84_32b34b39-946f-4c7b-855b-7905b1b84903/serve-healthcheck-canary/0.log"
Jul 27 19:14:54.120412 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:54.120372    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-jvqln_ab6797ac-2bcb-45b5-94a0-6359caeff863/insights-operator/0.log"
Jul 27 19:14:54.125520 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:54.125496    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-jvqln_ab6797ac-2bcb-45b5-94a0-6359caeff863/insights-operator/1.log"
Jul 27 19:14:54.143981 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:54.143956    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9jp2v_7275deb5-c992-47c5-91fb-eb478fba9a30/kube-rbac-proxy/0.log"
Jul 27 19:14:54.166411 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:54.166385    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9jp2v_7275deb5-c992-47c5-91fb-eb478fba9a30/exporter/0.log"
Jul 27 19:14:54.190175 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:54.190115    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9jp2v_7275deb5-c992-47c5-91fb-eb478fba9a30/extractor/0.log"
Jul 27 19:14:55.719620 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:55.719586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-r8n2x/perf-node-gather-daemonset-zxlrz"
Jul 27 19:14:59.105366 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:59.105332    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:14:59.166349 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:14:59.166315    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:15:00.564565 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:00.564526    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bfqq_f124ae20-ee26-472a-b3ae-65873226c9dd/discovery/0.log"
Jul 27 19:15:00.627618 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:00.627586    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b46967c9f-ptz2d_7c2bc8d6-6d7a-4453-8c15-129def8d3bea/router/0.log"
Jul 27 19:15:05.176549 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.176522    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/kube-multus-additional-cni-plugins/0.log"
Jul 27 19:15:05.200412 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.200380    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/egress-router-binary-copy/0.log"
Jul 27 19:15:05.220357 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.220332    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/cni-plugins/0.log"
Jul 27 19:15:05.241247 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.241217    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/bond-cni-plugin/0.log"
Jul 27 19:15:05.264591 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.264566    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/routeoverride-cni/0.log"
Jul 27 19:15:05.287714 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.287679    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/whereabouts-cni-bincopy/0.log"
Jul 27 19:15:05.308006 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.307984    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bpmlj_9a0a41fe-56a4-4dd0-90e0-5ba8ab992da8/whereabouts-cni/0.log"
Jul 27 19:15:05.670844 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.670805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gdzng_48c0dcb3-16e3-46a1-ba47-a91186ce974b/kube-multus/0.log"
Jul 27 19:15:05.786280 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.786250    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29lnk_c7d5444a-98d9-4fe7-8017-a35216c24332/network-metrics-daemon/0.log"
Jul 27 19:15:05.840231 ip-10-0-135-40 kubenswrapper[2577]: I0727 19:15:05.840200    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29lnk_c7d5444a-98d9-4fe7-8017-a35216c24332/kube-rbac-proxy/0.log"