Jul 09 18:01:14.828150 ip-10-0-143-79 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 18:01:14.828160 ip-10-0-143-79 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 18:01:14.828167 ip-10-0-143-79 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 18:01:14.828399 ip-10-0-143-79 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 18:01:25.019029 ip-10-0-143-79 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 18:01:25.019047 ip-10-0-143-79 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8381733a68154c03a666362de4a15b45 --
Jul 09 18:03:25.954468 ip-10-0-143-79 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 18:03:26.387598 ip-10-0-143-79 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:03:26.387598 ip-10-0-143-79 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 18:03:26.387598 ip-10-0-143-79 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:03:26.387598 ip-10-0-143-79 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 18:03:26.387598 ip-10-0-143-79 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:03:26.387598 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.383411    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390281    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390296    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390302    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390306    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390311    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390315    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390319    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390324    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390328    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390334    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390339    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390344    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390348    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390353    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390360    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390366    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390371    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390376    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:26.390435 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390382    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390388    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390394    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390398    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390403    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390408    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390414    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390418    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390423    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390427    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390432    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390436    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390440    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390445    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390449    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390454    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390458    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390462    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390466    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390470    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:26.391350 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390475    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390479    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390484    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390492    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390498    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390503    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390507    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390511    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390515    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390520    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390524    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390528    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390532    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390537    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390541    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390545    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390549    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390553    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390558    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390563    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:26.392258 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390568    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390572    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390576    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390580    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390584    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390589    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390593    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390598    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390602    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390606    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390610    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390615    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390619    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390624    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390628    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390633    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390638    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390643    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390647    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:26.393464 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390651    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390655    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390660    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390666    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390671    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390675    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390679    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390683    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390687    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390691    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390696    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390700    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390705    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390710    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390715    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390719    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390723    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390728    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390732    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:26.394556 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390737    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390741    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.390746    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.391987    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.391997    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392002    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392007    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392011    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392015    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392019    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392024    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392036    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392041    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392045    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392049    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392054    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392058    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392062    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392066    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392070    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:26.395399 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392075    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392080    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392084    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392088    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392094    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392098    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392102    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392106    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392110    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392115    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392118    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392123    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392127    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392132    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392140    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392146    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392151    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392156    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392160    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:26.396063 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392165    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392169    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392174    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392179    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392204    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392218    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392223    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392227    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392231    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392235    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392240    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392245    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392250    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392254    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392258    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392262    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392266    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392270    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392275    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392279    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:26.396889 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392283    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392287    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392294    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392301    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392308    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392314    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392318    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392322    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392327    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392331    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392336    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392340    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392344    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392348    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392353    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392357    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392360    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392364    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392370    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:26.399653 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392374    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392380    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392384    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392391    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392395    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392400    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392404    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392408    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392412    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392417    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392421    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392425    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392429    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392434    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392438    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392442    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392446    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392450    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392454    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392459    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:26.400299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392463    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392467    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392471    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.392475    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393089    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393098    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393103    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393108    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393112    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393117    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393121    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393128    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393133    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393137    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393142    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393147    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393150    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393154    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393158    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:26.400949 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393161    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393164    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393168    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393171    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393174    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393177    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393198    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393203    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393208    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393212    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393215    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393218    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393222    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393225    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393228    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393231    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393234    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393238    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393241    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393245    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:26.402005 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393248    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393252    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393255    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393258    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393261    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393265    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393268    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393271    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393274    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393277    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393281    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393284    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393289    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393293    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393297    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393300    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393305    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393308    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393311    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393315    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:26.402645 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393318    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393322    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393325    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393328    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393331    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393335    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393338    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393341    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393344    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393350    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393354    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393357    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393360    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393366    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393370    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393374    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393378    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393382    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393386    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:26.403486 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393389    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393393    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393397    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393401    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393404    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393408    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393411    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393415    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393419    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393423    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393427    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393431    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393435    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393439    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393443    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393447    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393451    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393455    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393459    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393463    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:26.404131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393467    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393471    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393477    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393483    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.393489    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393589    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393600    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393610    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393617    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393624    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393629    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393636    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393643    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393649    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393653    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393659    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393664    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393669    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393674    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393679    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393684    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393688    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393693    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 18:03:26.404928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393700    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393705    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393709    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393714    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393720    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393726    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393730    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393735    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393741    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393745    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393750    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393757    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393762    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393767    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393775    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393779    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393785    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393789    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393794    2581 flags.go:64] FLAG: --enable-server="true"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393799    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393806    2581 flags.go:64] FLAG: --event-burst="100"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393811    2581 flags.go:64] FLAG: --event-qps="50"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393817    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393823    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 18:03:26.405736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393828    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393834    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393839    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393843    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393848    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393853    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393858    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393863    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393870    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393874    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393880    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393885    2581 flags.go:64] FLAG: --feature-gates=""
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393891    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393896    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393900    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393905    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393910    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393915    2581 flags.go:64] FLAG: --help="false"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393919    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-143-79.ec2.internal"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393926    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393931    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393936    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393941    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 18:03:26.406407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393947    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393952    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393956    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393961    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393965    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393970    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393975    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393979    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393985    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393989    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393994    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.393999    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394003    2581 flags.go:64] FLAG: --lock-file=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394008    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394012    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394017    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394026    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394032    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394037    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394041    2581 flags.go:64] FLAG: --logging-format="text"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394046    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394051    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394056    2581 flags.go:64] FLAG: --manifest-url=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394061    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394067    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 18:03:26.407083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394072    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394078    2581 flags.go:64] FLAG: --max-pods="110"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394083    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394091    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394095    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394100    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394105    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394109    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394114    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394125    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394130    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394135    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394139    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394144    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394153    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394157    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394162    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394168    2581 flags.go:64] FLAG: --port="10250"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394172    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394177    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0101c9b0b94303cff"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394199    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394205    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394210    2581 flags.go:64] FLAG: --register-node="true"
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394215    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 09 18:03:26.465719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394221    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 09 18:03:26.462567 ip-10-0-143-79 systemd[1]: Started Kubernetes Kubelet.
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394228    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394233    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394237    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394243    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394248    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394253    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394257    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394262    2581 flags.go:64] FLAG: --runonce="false"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394267    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394271    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394277    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394282    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394287    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394291    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394295    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394299    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394303    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394307    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394310    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394314    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394318    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394322    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394325    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394332    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 18:03:26.467974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394337    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394340    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394346    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394349    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394353    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394356    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394360    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394364    2581 flags.go:64] FLAG: --v="2"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394370    2581 flags.go:64] FLAG: --version="false"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394376    2581 flags.go:64] FLAG: --vmodule=""
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394380    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.394384    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394540    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394546    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394552    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394557    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394561    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394566    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394572    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394576    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394582    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394589    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:26.469054 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394594    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394598    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394603    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394607    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394611    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394616    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394620    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394624    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394628    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394632    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394636    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394642    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394646    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394650    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394654    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394659    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394663    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394667    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394673    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:26.470120 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394677    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394682    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394686    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394690    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394694    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394698    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394703    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394707    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394713    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394724    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394728    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394734    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394738    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394743    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394747    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394751    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394757    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394761    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394766    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394770    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:26.470780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394774    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394779    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394783    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394787    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394791    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394796    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394801    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394805    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394809    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394813    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394818    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394823    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394827    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394831    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394836    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394840    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394847    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394852    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394857    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394861    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:26.471912 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394866    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394872    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394876    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394881    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394885    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394889    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394893    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394897    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394901    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394906    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394910    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394914    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394918    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394923    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394927    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394931    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394936    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394940    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394945    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394949    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:26.472610 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394954    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394958    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394962    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394968    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394972    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394976    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394980    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394984    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394989    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.394993    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.395001    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.406462    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.406478    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.406498    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 18:03:26.474178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.406504    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406565    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406570    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406574    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406577    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406580    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406583    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406586    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406589    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406592    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406594    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406597    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406600    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406604    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406608    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406611    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406614    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406617    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406620    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:26.474794 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406622    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406625    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406628    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406631    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406633    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406636    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406639    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406642    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406649    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406653    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406656    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406659    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406661    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406664    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406667    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406669    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406672    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406675    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406677    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406680    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:26.476482 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406683    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406685    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406688    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406690    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406693    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406696    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406698    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406701    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406703    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406706    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406709    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406712    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406715    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406718    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406721    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406723    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406726    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406728    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406731    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406734    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:26.477131 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406736    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406739    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406742    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406745    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406747    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406750    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406753    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406757    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406760    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406764    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406767    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406770    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406772    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406775    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406778    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406781    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406783    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406786    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406788    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406791    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:26.478199 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406794    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406797    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406800    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406804    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406808    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406811    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406814    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406817    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406820    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406823    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406826    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406830    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406832    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406835    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406839    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406842    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406845    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406847    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406850    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:26.478823 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406852    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406855    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.406860    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406981    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406987    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406991    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406994    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406997    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.406999    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407002    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407004    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407007    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407010    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407012    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407015    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:03:26.479447 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407018    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407021    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407024    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407026    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407029    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407031    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407034    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407037    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407040    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407042    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407045    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407047    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407052    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407056    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407059    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407062    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407064    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407067    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407070    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407073    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:03:26.480071 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407076    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407078    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407081    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407084    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407086    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407089    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407093    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407095    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407098    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407101    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407104    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407106    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407109    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407112    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407114    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407117    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407119    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407122    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407125    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:03:26.481026 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407127    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407130    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407132    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407135    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407137    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407140    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407143    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407145    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407148    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407150    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407154    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407157    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407161    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407163    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407166    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407169    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407172    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407174    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407177    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407180    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:03:26.482161 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407197    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407200    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407203    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407206    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407209    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407211    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407214    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407218    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407220    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407223    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407226    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407229    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407232    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407234    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407237    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407240    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407242    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407245    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407248    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:03:26.483708 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407251    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407254    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407256    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407260    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407264    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407267    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407269    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407272    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:26.407274    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.407279    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.407386    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.410001    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.411456    2581 server.go:1019] "Starting client certificate rotation"
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.411581    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:03:26.484600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.411608    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.414791    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.419675    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.429050    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.431411    2581 log.go:25] "Validated CRI v1 image API"
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.432041    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.436645    2581 fs.go:135] Filesystem UUIDs: map[4cfda11e-9715-458f-8092-e279ba961f29:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f3afb7d7-c4dc-4e75-99a4-d52fa4c14910:/dev/nvme0n1p4]
Jul 09 18:03:26.491993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.436670    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.443716    2581 manager.go:216] Machine: {Timestamp:2026-07-09 18:03:26.44311469 +0000 UTC m=+0.366659153 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3066679 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec200c6a51b8e42e538335483d6a650e SystemUUID:ec200c6a-51b8-e42e-5383-35483d6a650e BootID:8381733a-6815-4c03-a666-362de4a15b45 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:89:14:08:be:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:89:14:08:be:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:5a:bd:f9:70:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.443825    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.443912    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.445260    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.445285    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-79.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.446085    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.446094    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.446102    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.446987    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.448370    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451150    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451310    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451876    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451886    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451897    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451906    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.451915    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 18:03:26.557889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.452309    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.452333    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.454701    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.456801    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.456830    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.456983    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.456997    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457003    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457009    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457017    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457023    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457618    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457626    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457633    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457646    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457659    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.457668    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.458622    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.458630    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.461865    2581 server.go:1268] "Started kubelet"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.462779    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.463156    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.463217    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.464601    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.465392    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.468270    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.468699    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18c0b120b45dfc6f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-07-09 18:03:26.461844591 +0000 UTC m=+0.385389055,LastTimestamp:2026-07-09 18:03:26.461844591 +0000 UTC m=+0.385389055,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.469922    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-79.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.470054    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.470076    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.470144    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.470917    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.473422    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.474079    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.474095    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.474179    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.481114    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.481128    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 18:03:26.559107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.481555    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.481680    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.481746    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.482599    2581 factory.go:55] Registering systemd factory
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.492765    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.493074    2581 factory.go:153] Registering CRI-O factory
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.493088    2581 factory.go:223] Registration of the crio container factory successfully
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.493115    2581 factory.go:103] Registering Raw factory
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.493124    2581 manager.go:1196] Started watching for new ooms in manager
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.493540    2581 manager.go:349] Starting recovery of all containers
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.501883    2581 manager.go:354] Recovery completed
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.523346    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.528999    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.529045    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.529059    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.529630    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.529643    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.529660    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.529667    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.530775    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.530802    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.530808    2581 policy_none.go:49] "None policy: Start"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.530821    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.530822    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.530960    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.531097    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18c0b120b85f2e95  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-07-09 18:03:26.529031829 +0000 UTC m=+0.452576279,LastTimestamp:2026-07-09 18:03:26.529031829 +0000 UTC m=+0.452576279,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.532275    2581 policy_none.go:47] "Start"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.541916    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18c0b120b85f777b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-07-09 18:03:26.529050491 +0000 UTC m=+0.452594954,LastTimestamp:2026-07-09 18:03:26.529050491 +0000 UTC m=+0.452594954,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.543907    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:03:26.561502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.543994    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cx45q"
Jul 09 18:03:26.562694 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.561766    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-79.ec2.internal.18c0b120b85fa65a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-79.ec2.internal,UID:ip-10-0-143-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-79.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-79.ec2.internal,},FirstTimestamp:2026-07-09 18:03:26.52906249 +0000 UTC m=+0.452606953,LastTimestamp:2026-07-09 18:03:26.52906249 +0000 UTC m=+0.452606953,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-79.ec2.internal,}"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.573859    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.580966    2581 manager.go:337] "Starting Device Plugin manager"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.581014    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.581178    2581 server.go:87] "Starting device plugin registration server"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.581444    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.581457    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.581579    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.581692    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.581702    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.582284    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.582345    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.682717    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.683806    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.683839    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.683850    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.683880    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-79.ec2.internal"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.690329    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-79.ec2.internal"
Jul 09 18:03:26.697090 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.690348    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-79.ec2.internal\": node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.708970    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.791883    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.793166    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.793198    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.793220    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.793277    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.796778    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:26.824536 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.809620    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.898853    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"]
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.898935    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.900000    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.900029    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.900039    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.901464    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.901590    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.901616    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.902292    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.902324    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.902335    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.902292    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.902407    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.902421    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.903676    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.903703    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.905798    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.905832    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.905847    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.909887    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.923822    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-79.ec2.internal\" not found" node="ip-10-0-143-79.ec2.internal"
Jul 09 18:03:26.965946 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:26.927515    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-79.ec2.internal\" not found" node="ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.983444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7199f0b4371cb82c4658b72f79aae915-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7199f0b4371cb82c4658b72f79aae915\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.983711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7199f0b4371cb82c4658b72f79aae915-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7199f0b4371cb82c4658b72f79aae915\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:26.984859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/28e83951a50f7c287e9df4e6d546594a-config\") pod \"kube-apiserver-proxy-ip-10-0-143-79.ec2.internal\" (UID: \"28e83951a50f7c287e9df4e6d546594a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.010819    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.085309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7199f0b4371cb82c4658b72f79aae915-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7199f0b4371cb82c4658b72f79aae915\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.085252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7199f0b4371cb82c4658b72f79aae915-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7199f0b4371cb82c4658b72f79aae915\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.085376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7199f0b4371cb82c4658b72f79aae915-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7199f0b4371cb82c4658b72f79aae915\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.085396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/28e83951a50f7c287e9df4e6d546594a-config\") pod \"kube-apiserver-proxy-ip-10-0-143-79.ec2.internal\" (UID: \"28e83951a50f7c287e9df4e6d546594a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.085442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/28e83951a50f7c287e9df4e6d546594a-config\") pod \"kube-apiserver-proxy-ip-10-0-143-79.ec2.internal\" (UID: \"28e83951a50f7c287e9df4e6d546594a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.093416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.085475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7199f0b4371cb82c4658b72f79aae915-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal\" (UID: \"7199f0b4371cb82c4658b72f79aae915\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.116719 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.116694    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.217516 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.217474    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.226334 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.226317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.231672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.231518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.319820 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.319776    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.411151 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.411087    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 18:03:27.411589 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.411292    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 18:03:27.420496 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.420469    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.485668 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.485639    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:27.522275 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.522244    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.565697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.565679    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:03:27.565782 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.565689    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:58:27 +0000 UTC" deadline="2027-12-03 13:44:51.485020496 +0000 UTC"
Jul 09 18:03:27.565782 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.565715    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12283h41m23.919310177s"
Jul 09 18:03:27.566734 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.566718    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:03:27.566802 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.566746    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:03:27.566802 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.566762    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:03:27.623578 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:27.623548    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-79.ec2.internal\" not found"
Jul 09 18:03:27.659176 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.659152    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:27.684697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.684642    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:27.774618 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.774592    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.787489 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.787466    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:03:27.788889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.788873    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal"
Jul 09 18:03:27.802107 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.802086    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:03:27.890775 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:27.890743    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7199f0b4371cb82c4658b72f79aae915.slice/crio-ba1a7e704c7d1393d56b38488db10a5a822686b7363584a2dd884208739e7a8f WatchSource:0}: Error finding container ba1a7e704c7d1393d56b38488db10a5a822686b7363584a2dd884208739e7a8f: Status 404 returned error can't find the container with id ba1a7e704c7d1393d56b38488db10a5a822686b7363584a2dd884208739e7a8f
Jul 09 18:03:27.891257 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:27.891233    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e83951a50f7c287e9df4e6d546594a.slice/crio-e9320c0730f9b14afb1bcba8bc20c75c6b594834fd9b492bcbc323f4176da88e WatchSource:0}: Error finding container e9320c0730f9b14afb1bcba8bc20c75c6b594834fd9b492bcbc323f4176da88e: Status 404 returned error can't find the container with id e9320c0730f9b14afb1bcba8bc20c75c6b594834fd9b492bcbc323f4176da88e
Jul 09 18:03:27.895593 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:27.895573    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:03:28.455065 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.455029    2581 apiserver.go:52] "Watching apiserver"
Jul 09 18:03:28.465179 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.465150    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 18:03:28.465580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.465555    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-8lfv8","openshift-multus/multus-additional-cni-plugins-jpjpd","openshift-network-operator/iptables-alerter-b2dms","kube-system/konnectivity-agent-44zx7","kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl","openshift-image-registry/node-ca-wn8fk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal","openshift-multus/network-metrics-daemon-ghvbn","openshift-network-diagnostics/network-check-target-m42st","openshift-ovn-kubernetes/ovnkube-node-hvbkx","kube-system/global-pull-secret-syncer-7c7ll","openshift-cluster-node-tuning-operator/tuned-xfltw","openshift-dns/node-resolver-8kwh5"]
Jul 09 18:03:28.467146 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.467120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.467552 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.467413    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:03:28.468779 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.468754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.469786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.469514    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 18:03:28.469786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.469592    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w47d9\""
Jul 09 18:03:28.469786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.469671    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.469786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.469724    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.470054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.470006    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.471000 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.470850    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 18:03:28.471282 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.471238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.472328 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.472076    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 18:03:28.473037 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.472472    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j7xkq\""
Jul 09 18:03:28.473037 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.472959    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.473181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.473078    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.473291    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.473418    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.473591    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.474018    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.474216    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.474479    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vt9gw\""
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.474555    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.474775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.474698    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.475994 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.475421    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 18:03:28.475994 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.475561    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7n5kq\""
Jul 09 18:03:28.476337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.476317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.476583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.476563    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 18:03:28.477851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.477638    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.477851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.477715    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.477851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.477825    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-skzfs\""
Jul 09 18:03:28.479283 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.479248    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 18:03:28.479443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.479420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:28.479544 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.479495    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:28.479610 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.479558    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.479610 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.479562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9m72l\""
Jul 09 18:03:28.479766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.479748    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.480803 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.480782    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.482173 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.482154    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.482924 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.482904    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.483509 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.483488    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.483604 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.483534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:28.483604 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.483593    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:28.483973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.483805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dxt28\""
Jul 09 18:03:28.484645 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.484531    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 18:03:28.484753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.484679    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 18:03:28.484816 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.484782    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mc6dr\""
Jul 09 18:03:28.484816 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.484794    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 18:03:28.485090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.485060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.485203 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.485173    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 18:03:28.485604 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.485591    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 18:03:28.485752 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.485659    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:03:28.486669 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.486646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.486748 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.486723    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:28.487748 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.487728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4kzsw\""
Jul 09 18:03:28.487846 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.487733    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 18:03:28.491847 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-log-socket\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.491954 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovn-node-metrics-cert\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.491954 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7ac51175-6a26-4293-b812-30a20d95e941-iptables-alerter-script\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.491954 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6297deef-388a-44dc-8125-5a565307e957-host\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.491954 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-slash\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.491990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-cni-bin\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/022963e4-a05c-488f-b0c2-b8dbd79305eb-tmp-dir\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6297deef-388a-44dc-8125-5a565307e957-serviceca\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-run-netns\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dptk\" (UniqueName: \"kubernetes.io/projected/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-kube-api-access-7dptk\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b54996e8-faeb-4a8a-bc84-9ab7e37276db-konnectivity-ca\") pod \"konnectivity-agent-44zx7\" (UID: \"b54996e8-faeb-4a8a-bc84-9ab7e37276db\") " pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8gwx\" (UniqueName: \"kubernetes.io/projected/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-kube-api-access-j8gwx\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlfmw\" (UniqueName: \"kubernetes.io/projected/6297deef-388a-44dc-8125-5a565307e957-kube-api-access-dlfmw\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-kubelet\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-ovn\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-node-log\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-cni-netd\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-socket-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovnkube-config\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovnkube-script-lib\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.492497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmgqr\" (UniqueName: \"kubernetes.io/projected/7ac51175-6a26-4293-b812-30a20d95e941-kube-api-access-fmgqr\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b54996e8-faeb-4a8a-bc84-9ab7e37276db-agent-certs\") pod \"konnectivity-agent-44zx7\" (UID: \"b54996e8-faeb-4a8a-bc84-9ab7e37276db\") " pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492529    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-device-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-etc-selinux\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-sys-fs\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-env-overrides\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/022963e4-a05c-488f-b0c2-b8dbd79305eb-hosts-file\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-systemd-units\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-systemd\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-var-lib-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-etc-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7tmn\" (UniqueName: \"kubernetes.io/projected/022963e4-a05c-488f-b0c2-b8dbd79305eb-kube-api-access-m7tmn\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-registration-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-run-ovn-kubernetes\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.493095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.492919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ac51175-6a26-4293-b812-30a20d95e941-host-slash\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.515235 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.515189    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-skw4t"
Jul 09 18:03:28.522451 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.522430    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-skw4t"
Jul 09 18:03:28.575356 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.575334    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-conf-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-daemon-config\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovn-node-metrics-cert\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7ac51175-6a26-4293-b812-30a20d95e941-iptables-alerter-script\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysctl-conf\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6297deef-388a-44dc-8125-5a565307e957-host\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-cni-bin\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1c19302-1cda-47c9-94aa-ac799575abe2-kubelet-config\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp6jh\" (UniqueName: \"kubernetes.io/projected/9cdbabe3-1863-406c-bf94-d8d9933b364e-kube-api-access-rp6jh\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.596931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-systemd\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0aa3f04f-168f-4dfa-aa81-7fb702869305-tmp\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6297deef-388a-44dc-8125-5a565307e957-serviceca\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-run-netns\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b54996e8-faeb-4a8a-bc84-9ab7e37276db-konnectivity-ca\") pod \"konnectivity-agent-44zx7\" (UID: \"b54996e8-faeb-4a8a-bc84-9ab7e37276db\") " pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.598291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bghhq\" (UniqueName: \"kubernetes.io/projected/0aa3f04f-168f-4dfa-aa81-7fb702869305-kube-api-access-bghhq\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cnibin\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-system-cni-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8gwx\" (UniqueName: \"kubernetes.io/projected/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-kube-api-access-j8gwx\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-kubelet\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-ovn\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-node-log\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-cni-netd\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovnkube-config\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-os-release\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-cni-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmgqr\" (UniqueName: \"kubernetes.io/projected/7ac51175-6a26-4293-b812-30a20d95e941-kube-api-access-fmgqr\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-sys\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-system-cni-dir\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/452a82e7-5977-4a7e-9449-7fc238ecd9fd-cni-binary-copy\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.599299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.597987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-device-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-sys-fs\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-env-overrides\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-tuned\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-socket-dir-parent\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-systemd\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-etc-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-cni-bin\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysconfig\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-run\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-log-socket\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598770    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598905    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-797b5\" (UniqueName: \"kubernetes.io/projected/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-kube-api-access-797b5\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-kubelet\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.598972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-hostroot\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-multus-certs\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-slash\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1c19302-1cda-47c9-94aa-ac799575abe2-dbus\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-kubernetes\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/022963e4-a05c-488f-b0c2-b8dbd79305eb-tmp-dir\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dptk\" (UniqueName: \"kubernetes.io/projected/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-kube-api-access-7dptk\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-host\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-cni-multus\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlfmw\" (UniqueName: \"kubernetes.io/projected/6297deef-388a-44dc-8125-5a565307e957-kube-api-access-dlfmw\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b54996e8-faeb-4a8a-bc84-9ab7e37276db-agent-certs\") pod \"konnectivity-agent-44zx7\" (UID: \"b54996e8-faeb-4a8a-bc84-9ab7e37276db\") " pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-log-socket\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-var-lib-kubelet\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599704    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/022963e4-a05c-488f-b0c2-b8dbd79305eb-tmp-dir\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cni-binary-copy\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-systemd\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.599789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-cnibin\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.600896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovnkube-config\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-env-overrides\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-sys-fs\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600288    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-etc-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-node-log\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-slash\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6297deef-388a-44dc-8125-5a565307e957-host\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-cni-bin\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600551    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6297deef-388a-44dc-8125-5a565307e957-serviceca\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-run-netns\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.601085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b54996e8-faeb-4a8a-bc84-9ab7e37276db-konnectivity-ca\") pod \"konnectivity-agent-44zx7\" (UID: \"b54996e8-faeb-4a8a-bc84-9ab7e37276db\") " pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.600397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-etc-kubernetes\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.601749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.601681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7ac51175-6a26-4293-b812-30a20d95e941-iptables-alerter-script\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.602484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.601764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-cni-netd\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.602484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.601871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-socket-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.602484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.601878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-device-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.602484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-run-ovn\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.602484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-kubelet\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.602484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovnkube-script-lib\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.602778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602538    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-socket-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.602778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-modprobe-d\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.602778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-netns\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.602973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4qcn\" (UniqueName: \"kubernetes.io/projected/452a82e7-5977-4a7e-9449-7fc238ecd9fd-kube-api-access-v4qcn\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.602973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.602899    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-etc-selinux\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.603286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.603286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovnkube-script-lib\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.603286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-os-release\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.603286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-etc-selinux\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.603286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-k8s-cni-cncf-io\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/022963e4-a05c-488f-b0c2-b8dbd79305eb-hosts-file\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-systemd-units\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/022963e4-a05c-488f-b0c2-b8dbd79305eb-hosts-file\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-var-lib-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-systemd-units\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:28.603638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-var-lib-openvswitch\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.603992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysctl-d\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.603992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603776    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7tmn\" (UniqueName: \"kubernetes.io/projected/022963e4-a05c-488f-b0c2-b8dbd79305eb-kube-api-access-m7tmn\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.603992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.603941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-registration-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.604148 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.604029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-run-ovn-kubernetes\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.604148 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.604067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-registration-dir\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.604148 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.604132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ac51175-6a26-4293-b812-30a20d95e941-host-slash\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.604331 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.604242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-host-run-ovn-kubernetes\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.604331 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.604265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-lib-modules\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.604331 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.604305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7ac51175-6a26-4293-b812-30a20d95e941-host-slash\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.606531 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.606469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-ovn-node-metrics-cert\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.606650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.606589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b54996e8-faeb-4a8a-bc84-9ab7e37276db-agent-certs\") pod \"konnectivity-agent-44zx7\" (UID: \"b54996e8-faeb-4a8a-bc84-9ab7e37276db\") " pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.611406 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.611378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8gwx\" (UniqueName: \"kubernetes.io/projected/84b3abce-5e4b-4cf1-b60a-2dfd905a6c67-kube-api-access-j8gwx\") pod \"aws-ebs-csi-driver-node-6cgzl\" (UID: \"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.612078 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.612042    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmgqr\" (UniqueName: \"kubernetes.io/projected/7ac51175-6a26-4293-b812-30a20d95e941-kube-api-access-fmgqr\") pod \"iptables-alerter-b2dms\" (UID: \"7ac51175-6a26-4293-b812-30a20d95e941\") " pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.612742 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.612716    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlfmw\" (UniqueName: \"kubernetes.io/projected/6297deef-388a-44dc-8125-5a565307e957-kube-api-access-dlfmw\") pod \"node-ca-wn8fk\" (UID: \"6297deef-388a-44dc-8125-5a565307e957\") " pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.612742 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.612724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dptk\" (UniqueName: \"kubernetes.io/projected/ffdf63a8-f00b-4da3-b68c-e45a92e1af50-kube-api-access-7dptk\") pod \"ovnkube-node-hvbkx\" (UID: \"ffdf63a8-f00b-4da3-b68c-e45a92e1af50\") " pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.614775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.614728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7tmn\" (UniqueName: \"kubernetes.io/projected/022963e4-a05c-488f-b0c2-b8dbd79305eb-kube-api-access-m7tmn\") pod \"node-resolver-8kwh5\" (UID: \"022963e4-a05c-488f-b0c2-b8dbd79305eb\") " pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.705337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-cni-bin\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysconfig\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.705337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-run\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.705337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-cni-bin\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysconfig\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-797b5\" (UniqueName: \"kubernetes.io/projected/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-kube-api-access-797b5\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-run\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-kubelet\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-kubelet\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-hostroot\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-multus-certs\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-hostroot\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1c19302-1cda-47c9-94aa-ac799575abe2-dbus\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-kubernetes\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-multus-certs\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.705675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-host\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-kubernetes\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-cni-multus\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1c19302-1cda-47c9-94aa-ac799575abe2-dbus\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-var-lib-kubelet\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cni-binary-copy\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-var-lib-cni-multus\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-host\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-cnibin\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-etc-kubernetes\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-var-lib-kubelet\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-modprobe-d\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-cnibin\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-netns\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-etc-kubernetes\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4qcn\" (UniqueName: \"kubernetes.io/projected/452a82e7-5977-4a7e-9449-7fc238ecd9fd-kube-api-access-v4qcn\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.705974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.706402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-os-release\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706046    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-modprobe-d\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-k8s-cni-cncf-io\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-k8s-cni-cncf-io\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysctl-d\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-lib-modules\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-conf-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-daemon-config\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysctl-conf\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1c19302-1cda-47c9-94aa-ac799575abe2-kubelet-config\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp6jh\" (UniqueName: \"kubernetes.io/projected/9cdbabe3-1863-406c-bf94-d8d9933b364e-kube-api-access-rp6jh\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-systemd\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0aa3f04f-168f-4dfa-aa81-7fb702869305-tmp\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cni-binary-copy\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.707320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bghhq\" (UniqueName: \"kubernetes.io/projected/0aa3f04f-168f-4dfa-aa81-7fb702869305-kube-api-access-bghhq\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-host-run-netns\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cnibin\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1c19302-1cda-47c9-94aa-ac799575abe2-kubelet-config\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.706536    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-lib-modules\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysctl-conf\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-system-cni-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-conf-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.706641    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-os-release\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-cnibin\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.706751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:03:29.206715812 +0000 UTC m=+3.130260294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706791    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-system-cni-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-cni-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-sys\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-system-cni-dir\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:28.708162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/452a82e7-5977-4a7e-9449-7fc238ecd9fd-cni-binary-copy\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.706963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-sysctl-d\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-tuned\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707014    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-systemd\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.707035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:29.207017682 +0000 UTC m=+3.130562150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-socket-dir-parent\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707107    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0aa3f04f-168f-4dfa-aa81-7fb702869305-sys\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-os-release\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707136    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-socket-dir-parent\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-system-cni-dir\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-os-release\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-cni-dir\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/452a82e7-5977-4a7e-9449-7fc238ecd9fd-cni-binary-copy\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.709145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.707816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/452a82e7-5977-4a7e-9449-7fc238ecd9fd-multus-daemon-config\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.710061 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.710041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0aa3f04f-168f-4dfa-aa81-7fb702869305-tmp\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.710796 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.710770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0aa3f04f-168f-4dfa-aa81-7fb702869305-etc-tuned\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.714066 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.714042    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:03:28.714066 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.714068    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:03:28.714228 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.714083    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:28.714228 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:28.714156    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:29.214136157 +0000 UTC m=+3.137680622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:28.714418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.714391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-797b5\" (UniqueName: \"kubernetes.io/projected/1b9f6d09-bbda-4c8d-a4c7-39a062fa1681-kube-api-access-797b5\") pod \"multus-additional-cni-plugins-jpjpd\" (UID: \"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681\") " pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.715805 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.715763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bghhq\" (UniqueName: \"kubernetes.io/projected/0aa3f04f-168f-4dfa-aa81-7fb702869305-kube-api-access-bghhq\") pod \"tuned-xfltw\" (UID: \"0aa3f04f-168f-4dfa-aa81-7fb702869305\") " pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.716311 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.716290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4qcn\" (UniqueName: \"kubernetes.io/projected/452a82e7-5977-4a7e-9449-7fc238ecd9fd-kube-api-access-v4qcn\") pod \"multus-8lfv8\" (UID: \"452a82e7-5977-4a7e-9449-7fc238ecd9fd\") " pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.716311 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.716255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp6jh\" (UniqueName: \"kubernetes.io/projected/9cdbabe3-1863-406c-bf94-d8d9933b364e-kube-api-access-rp6jh\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:28.781695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.781665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wn8fk"
Jul 09 18:03:28.789788 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.789534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:28.792601 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.792575    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6297deef_388a_44dc_8125_5a565307e957.slice/crio-6e8c733342703cf7751c9ff01520b5cde6ea96d8f1060bc1b3bf80a0313d0426 WatchSource:0}: Error finding container 6e8c733342703cf7751c9ff01520b5cde6ea96d8f1060bc1b3bf80a0313d0426: Status 404 returned error can't find the container with id 6e8c733342703cf7751c9ff01520b5cde6ea96d8f1060bc1b3bf80a0313d0426
Jul 09 18:03:28.798703 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.798636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wn8fk" event={"ID":"6297deef-388a-44dc-8125-5a565307e957","Type":"ContainerStarted","Data":"6e8c733342703cf7751c9ff01520b5cde6ea96d8f1060bc1b3bf80a0313d0426"}
Jul 09 18:03:28.800560 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.800534    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffdf63a8_f00b_4da3_b68c_e45a92e1af50.slice/crio-3381ee9c3ea677ba73ab2ce8601d11dbc55431a55037a087a504eae05881e031 WatchSource:0}: Error finding container 3381ee9c3ea677ba73ab2ce8601d11dbc55431a55037a087a504eae05881e031: Status 404 returned error can't find the container with id 3381ee9c3ea677ba73ab2ce8601d11dbc55431a55037a087a504eae05881e031
Jul 09 18:03:28.800672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.800579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" event={"ID":"7199f0b4371cb82c4658b72f79aae915","Type":"ContainerStarted","Data":"ba1a7e704c7d1393d56b38488db10a5a822686b7363584a2dd884208739e7a8f"}
Jul 09 18:03:28.800672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.800649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2dms"
Jul 09 18:03:28.802065 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.802037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal" event={"ID":"28e83951a50f7c287e9df4e6d546594a","Type":"ContainerStarted","Data":"e9320c0730f9b14afb1bcba8bc20c75c6b594834fd9b492bcbc323f4176da88e"}
Jul 09 18:03:28.807778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.807752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:28.810215 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.810188    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac51175_6a26_4293_b812_30a20d95e941.slice/crio-1c4a1a86cc259b0a5a35d297ba33bf6484dbbe6863c17aa52e3c6e848759ddf7 WatchSource:0}: Error finding container 1c4a1a86cc259b0a5a35d297ba33bf6484dbbe6863c17aa52e3c6e848759ddf7: Status 404 returned error can't find the container with id 1c4a1a86cc259b0a5a35d297ba33bf6484dbbe6863c17aa52e3c6e848759ddf7
Jul 09 18:03:28.816087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.816027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8kwh5"
Jul 09 18:03:28.817506 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.817481    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54996e8_faeb_4a8a_bc84_9ab7e37276db.slice/crio-b19e81343f5383586123e766d928f430f99f8acfb8d644d441bb210e87edf1fb WatchSource:0}: Error finding container b19e81343f5383586123e766d928f430f99f8acfb8d644d441bb210e87edf1fb: Status 404 returned error can't find the container with id b19e81343f5383586123e766d928f430f99f8acfb8d644d441bb210e87edf1fb
Jul 09 18:03:28.824880 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.824735    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl"
Jul 09 18:03:28.826185 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.826157    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod022963e4_a05c_488f_b0c2_b8dbd79305eb.slice/crio-b54f2714672b023b7c40d8545b21f0f52beecb94bd0348cb6b50f78e2d4e25ba WatchSource:0}: Error finding container b54f2714672b023b7c40d8545b21f0f52beecb94bd0348cb6b50f78e2d4e25ba: Status 404 returned error can't find the container with id b54f2714672b023b7c40d8545b21f0f52beecb94bd0348cb6b50f78e2d4e25ba
Jul 09 18:03:28.834360 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.834340    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xfltw"
Jul 09 18:03:28.834832 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.834795    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b3abce_5e4b_4cf1_b60a_2dfd905a6c67.slice/crio-244c1abac8684531f09a80f16931cc6f526d5a17b38322f310f771cffe00578d WatchSource:0}: Error finding container 244c1abac8684531f09a80f16931cc6f526d5a17b38322f310f771cffe00578d: Status 404 returned error can't find the container with id 244c1abac8684531f09a80f16931cc6f526d5a17b38322f310f771cffe00578d
Jul 09 18:03:28.841413 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.841228    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jpjpd"
Jul 09 18:03:28.842818 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.842795    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa3f04f_168f_4dfa_aa81_7fb702869305.slice/crio-1a4db2cc09766706b5a2a41e054b58056864ed60c1e4953ae4a02c2b76678cb7 WatchSource:0}: Error finding container 1a4db2cc09766706b5a2a41e054b58056864ed60c1e4953ae4a02c2b76678cb7: Status 404 returned error can't find the container with id 1a4db2cc09766706b5a2a41e054b58056864ed60c1e4953ae4a02c2b76678cb7
Jul 09 18:03:28.848819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:28.848794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lfv8"
Jul 09 18:03:28.853579 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.853549    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9f6d09_bbda_4c8d_a4c7_39a062fa1681.slice/crio-80ba245d2e27710474bee8da43f8e3b8e82a7015e2eaa362522f2830011d1b31 WatchSource:0}: Error finding container 80ba245d2e27710474bee8da43f8e3b8e82a7015e2eaa362522f2830011d1b31: Status 404 returned error can't find the container with id 80ba245d2e27710474bee8da43f8e3b8e82a7015e2eaa362522f2830011d1b31
Jul 09 18:03:28.860442 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:03:28.860413    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452a82e7_5977_4a7e_9449_7fc238ecd9fd.slice/crio-85e293255bc98fa2c2a39162e23404813ee11ce38251cf18db747b2d49fa38ae WatchSource:0}: Error finding container 85e293255bc98fa2c2a39162e23404813ee11ce38251cf18db747b2d49fa38ae: Status 404 returned error can't find the container with id 85e293255bc98fa2c2a39162e23404813ee11ce38251cf18db747b2d49fa38ae
Jul 09 18:03:29.211014 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.210975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:29.211346 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.211027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:29.211346 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.211105    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:29.211346 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.211153    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:29.211346 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.211194    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:30.21117018 +0000 UTC m=+4.134714638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:29.211346 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.211233    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:03:30.211220774 +0000 UTC m=+4.134765227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:29.312332 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.312292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:29.312537 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.312487    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:03:29.312537 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.312515    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:03:29.312537 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.312529    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:29.312701 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:29.312599    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:30.312578452 +0000 UTC m=+4.236122904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:29.525167 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.525004    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:58:29 +0000 UTC" deadline="2028-04-25 13:21:19.399838456 +0000 UTC"
Jul 09 18:03:29.525167 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.525066    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15739h17m49.874775827s"
Jul 09 18:03:29.805934 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.805760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerStarted","Data":"80ba245d2e27710474bee8da43f8e3b8e82a7015e2eaa362522f2830011d1b31"}
Jul 09 18:03:29.807859 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.807791    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xfltw" event={"ID":"0aa3f04f-168f-4dfa-aa81-7fb702869305","Type":"ContainerStarted","Data":"1a4db2cc09766706b5a2a41e054b58056864ed60c1e4953ae4a02c2b76678cb7"}
Jul 09 18:03:29.809038 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.809012    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8kwh5" event={"ID":"022963e4-a05c-488f-b0c2-b8dbd79305eb","Type":"ContainerStarted","Data":"b54f2714672b023b7c40d8545b21f0f52beecb94bd0348cb6b50f78e2d4e25ba"}
Jul 09 18:03:29.810369 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.810336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"3381ee9c3ea677ba73ab2ce8601d11dbc55431a55037a087a504eae05881e031"}
Jul 09 18:03:29.811605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.811544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl" event={"ID":"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67","Type":"ContainerStarted","Data":"244c1abac8684531f09a80f16931cc6f526d5a17b38322f310f771cffe00578d"}
Jul 09 18:03:29.812975 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.812931    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-44zx7" event={"ID":"b54996e8-faeb-4a8a-bc84-9ab7e37276db","Type":"ContainerStarted","Data":"b19e81343f5383586123e766d928f430f99f8acfb8d644d441bb210e87edf1fb"}
Jul 09 18:03:29.814280 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.814214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2dms" event={"ID":"7ac51175-6a26-4293-b812-30a20d95e941","Type":"ContainerStarted","Data":"1c4a1a86cc259b0a5a35d297ba33bf6484dbbe6863c17aa52e3c6e848759ddf7"}
Jul 09 18:03:29.815903 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:29.815857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lfv8" event={"ID":"452a82e7-5977-4a7e-9449-7fc238ecd9fd","Type":"ContainerStarted","Data":"85e293255bc98fa2c2a39162e23404813ee11ce38251cf18db747b2d49fa38ae"}
Jul 09 18:03:30.219831 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.219789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:30.220030 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.219848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:30.220110 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.220034    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:30.220110 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.220102    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:03:32.220081621 +0000 UTC m=+6.143626076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:30.220581 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.220533    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:30.220685 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.220589    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:32.220573213 +0000 UTC m=+6.144117665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:30.320587 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.320545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:30.320780 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.320740    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:03:30.320780 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.320760    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:03:30.320780 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.320773    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:30.320946 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.320835    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:32.320815563 +0000 UTC m=+6.244360016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:30.525959 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.525831    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:58:29 +0000 UTC" deadline="2028-03-11 04:18:55.632799891 +0000 UTC"
Jul 09 18:03:30.525959 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.525875    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14650h15m25.106930221s"
Jul 09 18:03:30.795932 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.795903    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:30.796080 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.796048    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:30.796839 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.796489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:30.796839 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.796601    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:30.796839 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:30.796667    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:30.796839 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:30.796743    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:31.827675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:31.827621    2581 generic.go:356] "Generic (PLEG): container finished" podID="7199f0b4371cb82c4658b72f79aae915" containerID="2ecc8a267b87ca936b79a9df5f5b0ed8b5a489b2aaa2500bcbf3cdc70f14ae65" exitCode=0
Jul 09 18:03:31.836289 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:31.835525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" event={"ID":"7199f0b4371cb82c4658b72f79aae915","Type":"ContainerDied","Data":"2ecc8a267b87ca936b79a9df5f5b0ed8b5a489b2aaa2500bcbf3cdc70f14ae65"}
Jul 09 18:03:31.840769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:31.840579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal" event={"ID":"28e83951a50f7c287e9df4e6d546594a","Type":"ContainerStarted","Data":"1640a98e0982f3fb3bbef1d765ad373b91444f94d988065670d5bfe1f5f87ad6"}
Jul 09 18:03:31.868891 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:31.868502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-79.ec2.internal" podStartSLOduration=3.868482124 podStartE2EDuration="3.868482124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:31.867819392 +0000 UTC m=+5.791363870" watchObservedRunningTime="2026-07-09 18:03:31.868482124 +0000 UTC m=+5.792026601"
Jul 09 18:03:32.242056 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.242015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:32.242215 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.242074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:32.242311 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.242286    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:32.242368 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.242358    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:03:36.242335671 +0000 UTC m=+10.165880127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:32.242785 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.242761    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:32.242865 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.242826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:36.242809697 +0000 UTC m=+10.166354152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:32.342540 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.342441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:32.342764 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.342743    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:03:32.342836 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.342771    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:03:32.342836 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.342786    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:32.342975 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.342961    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:36.342938443 +0000 UTC m=+10.266482897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:32.796663 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.795884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:32.796663 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.796018    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:32.796663 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.796374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:32.796663 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.796457    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:32.796663 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.796516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:32.796663 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:32.796573    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:32.850961 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:32.850865    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" event={"ID":"7199f0b4371cb82c4658b72f79aae915","Type":"ContainerStarted","Data":"37d44aba96b74ffcc18b75e8fe5a78b866cfe6aa1e034c4c2bbe8978fc760abc"}
Jul 09 18:03:34.794317 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:34.794276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:34.794812 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:34.794421    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:34.794812 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:34.794276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:34.794812 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:34.794470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:34.794812 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:34.794534    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:34.794812 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:34.794643    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:36.275331 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:36.275293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:36.275834 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:36.275509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:36.275834 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.275554    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:36.275834 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.275635    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:44.275610947 +0000 UTC m=+18.199155401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:36.275834 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.275698    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:36.275834 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.275782    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:03:44.275736715 +0000 UTC m=+18.199281167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:36.376764 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:36.376717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:36.376936 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.376918    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:03:36.376994 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.376938    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:03:36.376994 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.376952    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:36.377120 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.377012    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:03:44.376992654 +0000 UTC m=+18.300537108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:36.795217 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:36.794999    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:36.795217 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:36.795095    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:36.795217 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.795112    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:36.795567 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.795258    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:36.795567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:36.795341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:36.795567 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:36.795456    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:38.794035 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:38.793990    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:38.794566 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:38.794161    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:38.794682 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:38.794646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:38.794786 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:38.794750    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:38.795053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:38.794923    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:38.795053 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:38.795017    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:40.793584 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:40.793559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:40.793584 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:40.793575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:40.793986 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:40.793575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:40.793986 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:40.793656    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:40.793986 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:40.793728    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:40.793986 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:40.793789    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:42.793608 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:42.793576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:42.794085 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:42.793574    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:42.794085 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:42.793707    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:42.794085 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:42.793721    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:42.794085 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:42.793838    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:42.794085 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:42.793966    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:44.335808 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:44.335775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:44.336358 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:44.335863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:44.336358 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.335920    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:44.336358 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.335967    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:44.336358 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.335995    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:00.335972119 +0000 UTC m=+34.259516586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:03:44.336358 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.336031    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:00.336012066 +0000 UTC m=+34.259556527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:03:44.436894 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:44.436863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:44.437072 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.437052    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:03:44.437218 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.437079    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:03:44.437218 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.437092    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:44.437218 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.437153    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:00.437133312 +0000 UTC m=+34.360677785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:03:44.793691 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:44.793657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:44.793691 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:44.793687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:44.793919 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:44.793702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:44.793919 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.793798    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:44.793919 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.793888    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:44.794070 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:44.793942    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:46.795072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:46.795037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:46.795547 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:46.795143    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:46.795547 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:46.795268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:46.795547 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:46.795404    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:46.795547 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:46.795454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:46.795707 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:46.795579    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:48.793439 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:48.793397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:48.793439 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:48.793417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:48.793978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:48.793397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:48.793978 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:48.793516    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:48.793978 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:48.793581    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:48.793978 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:48.793653    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:49.886531 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.886414    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9f6d09-bbda-4c8d-a4c7-39a062fa1681" containerID="4586fc93fe6daf428b9555e4fbfc7d6923dcc65d64d0ee98bbe83769a1e126e3" exitCode=0
Jul 09 18:03:49.887027 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.886493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerDied","Data":"4586fc93fe6daf428b9555e4fbfc7d6923dcc65d64d0ee98bbe83769a1e126e3"}
Jul 09 18:03:49.888014 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.887986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xfltw" event={"ID":"0aa3f04f-168f-4dfa-aa81-7fb702869305","Type":"ContainerStarted","Data":"92ccee6d603225621c51673f869654d8dbf4c7aa144598ca48cfeda4cb5efa43"}
Jul 09 18:03:49.889534 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.889480    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8kwh5" event={"ID":"022963e4-a05c-488f-b0c2-b8dbd79305eb","Type":"ContainerStarted","Data":"c9bee45f3a4c719f5656809889587aafe39a78e97eb9737c6dd34d8f2141934f"}
Jul 09 18:03:49.891810 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.891793    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:03:49.892119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.892100    2581 generic.go:356] "Generic (PLEG): container finished" podID="ffdf63a8-f00b-4da3-b68c-e45a92e1af50" containerID="5559a9c63574e19489934f940ed8440be50d0be0198de7306f820f6dfc8b8943" exitCode=1
Jul 09 18:03:49.892196 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.892166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"fc6102b4a4fc077aca95d478adfbf66552e25a0e03d4a590a80b528c6c9be1b1"}
Jul 09 18:03:49.892247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.892234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"1ef10092251bf18d63c17ceb5ef125e7eff38e5cc5dd8fe4761652a2a8e5afcc"}
Jul 09 18:03:49.892284 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.892269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"ca6d4b56dc97b6a9f7681cd0251fc2cb00e7e16aeb639558b2fded4f7388fbdd"}
Jul 09 18:03:49.892316 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.892296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerDied","Data":"5559a9c63574e19489934f940ed8440be50d0be0198de7306f820f6dfc8b8943"}
Jul 09 18:03:49.892345 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.892325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"1044031b102212b4405ceb83e7c8f59f55bc10bf8e825743c8f5269199cb7e41"}
Jul 09 18:03:49.893556 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.893536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl" event={"ID":"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67","Type":"ContainerStarted","Data":"f02498f6afd471677d214aa860081d7b92f2eec5a7c0be9b5cf3fbafada7f1d5"}
Jul 09 18:03:49.894689 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.894671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-44zx7" event={"ID":"b54996e8-faeb-4a8a-bc84-9ab7e37276db","Type":"ContainerStarted","Data":"eb6d7fe83a0bbc450291f545a94cc48904117e3415ffb0b01dd38d5663c77fb2"}
Jul 09 18:03:49.896061 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.896043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wn8fk" event={"ID":"6297deef-388a-44dc-8125-5a565307e957","Type":"ContainerStarted","Data":"84ea52d1157dff8ab2408083d350a3fffc6672bc1c29c201eb803171802c3863"}
Jul 09 18:03:49.897397 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.897380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lfv8" event={"ID":"452a82e7-5977-4a7e-9449-7fc238ecd9fd","Type":"ContainerStarted","Data":"86e0739f2589a870dcc550cf846f11f1d6d1a0c56be36f81b000dfccc6795778"}
Jul 09 18:03:49.916314 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.916277    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-79.ec2.internal" podStartSLOduration=21.91626527 podStartE2EDuration="21.91626527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:32.869451392 +0000 UTC m=+6.792995870" watchObservedRunningTime="2026-07-09 18:03:49.91626527 +0000 UTC m=+23.839809741"
Jul 09 18:03:49.939100 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.939064    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8lfv8" podStartSLOduration=2.4262321350000002 podStartE2EDuration="22.939049702s" totalImagesPullingTime="20.512817567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:49.938884423 +0000 UTC m=+23.862428895" watchObservedRunningTime="2026-07-09 18:03:49.939049702 +0000 UTC m=+23.862594174"
Jul 09 18:03:49.982355 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.982301    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8kwh5" podStartSLOduration=2.467572537 podStartE2EDuration="22.982280025s" totalImagesPullingTime="20.514707488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:49.981332284 +0000 UTC m=+23.904876754" watchObservedRunningTime="2026-07-09 18:03:49.982280025 +0000 UTC m=+23.905824498"
Jul 09 18:03:49.982858 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:49.982824    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-44zx7" podStartSLOduration=2.4818042 podStartE2EDuration="22.982800405s" totalImagesPullingTime="20.500996205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:49.961913766 +0000 UTC m=+23.885458237" watchObservedRunningTime="2026-07-09 18:03:49.982800405 +0000 UTC m=+23.906344882"
Jul 09 18:03:50.027390 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.027348    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xfltw" podStartSLOduration=2.512037537 podStartE2EDuration="23.027333505s" totalImagesPullingTime="20.515295968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:50.000257155 +0000 UTC m=+23.923801647" watchObservedRunningTime="2026-07-09 18:03:50.027333505 +0000 UTC m=+23.950877973"
Jul 09 18:03:50.027707 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.027687    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wn8fk" podStartSLOduration=2.523153142 podStartE2EDuration="23.027681663s" totalImagesPullingTime="20.504528521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:50.02714119 +0000 UTC m=+23.950685662" watchObservedRunningTime="2026-07-09 18:03:50.027681663 +0000 UTC m=+23.951226135"
Jul 09 18:03:50.063487 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.063455    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:50.138497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.138386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:50.215105 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.215080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:50.794897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.794541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:50.794897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.794574    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:50.794897 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:50.794690    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:50.794897 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:50.794744    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:50.794897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.794769    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:50.794897 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:50.794856    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:50.901947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.901899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2dms" event={"ID":"7ac51175-6a26-4293-b812-30a20d95e941","Type":"ContainerStarted","Data":"9b702a1ecaf1a8b0a9ada3659ed7f48aa26496cd07e4e93b81b637def4386574"}
Jul 09 18:03:50.905218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.905176    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:03:50.905639 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:50.905605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"fa0bd3881fa13f2521cd631a0cd3bec4eabe3b3a27365003668831eb6015f6fc"}
Jul 09 18:03:51.014175 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.014134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:51.096481 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.096434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:51.145469 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.145440    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 18:03:51.186069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.186030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:51.255421 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.255391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:51.593975 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.593880    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:03:51.145458461Z","UUID":"a05e7ce2-33df-44a3-983f-9f61c28ae1af","Handler":null,"Name":"","Endpoint":""}
Jul 09 18:03:51.595726 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.595707    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 18:03:51.595726 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.595743    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 18:03:51.910000 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:51.909964    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl" event={"ID":"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67","Type":"ContainerStarted","Data":"f67e01b01ffce1d20e5b46980569abb927e3f51658c88315a8572ba55e86ad7d"}
Jul 09 18:03:52.025722 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.025688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:52.318431 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.318411    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:52.318903 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.318887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:52.334529 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.334483    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b2dms" podStartSLOduration=4.783254533 podStartE2EDuration="25.334469147s" totalImagesPullingTime="20.551214614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:50.91677726 +0000 UTC m=+24.840321746" watchObservedRunningTime="2026-07-09 18:03:52.334469147 +0000 UTC m=+26.258013614"
Jul 09 18:03:52.388255 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.388210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:52.446329 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.445081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:52.797578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.797386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:52.797754 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.797395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:52.797754 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:52.797663    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:52.797883 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:52.797771    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:52.797883 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.797406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:52.797959 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:52.797880    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:52.836746 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.836715    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:52.896707 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.896678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:52.916005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.915983    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:03:52.916509 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.916472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"3efc9495fbbb46c25aec2d431316ccf8a4dc7cf8426761b750e1010c0e06d354"}
Jul 09 18:03:52.916721 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.916701    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:52.917164 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.917146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-44zx7"
Jul 09 18:03:52.991391 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:52.991362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:54.794291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.794171    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:54.794291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.794177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:54.794967 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.794294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:54.794967 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:54.794295    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:54.794967 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:54.794380    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:54.794967 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:54.794472    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:54.921328 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.921292    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9f6d09-bbda-4c8d-a4c7-39a062fa1681" containerID="c631af29a992e8c6b4668095edb125274051df2964dfcd8749aef398c9df030f" exitCode=0
Jul 09 18:03:54.921480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.921377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerDied","Data":"c631af29a992e8c6b4668095edb125274051df2964dfcd8749aef398c9df030f"}
Jul 09 18:03:54.924999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.924981    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:03:54.925385 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.925364    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"e3543ddec40c7183904bdda64d3ea175817c062bf9e634bfa4e0e92064ea9b41"}
Jul 09 18:03:54.925568 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.925549    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:54.925766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.925744    2581 scope.go:117] "RemoveContainer" containerID="5559a9c63574e19489934f940ed8440be50d0be0198de7306f820f6dfc8b8943"
Jul 09 18:03:54.928423 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.928397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl" event={"ID":"84b3abce-5e4b-4cf1-b60a-2dfd905a6c67","Type":"ContainerStarted","Data":"f25189f2a3fe857103f2613a5468d7f66f70a54fc28bb643b3580febca5fddd8"}
Jul 09 18:03:54.942179 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.942163    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:54.960857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:54.960816    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6cgzl" podStartSLOduration=3.066510483 podStartE2EDuration="27.960800686s" totalImagesPullingTime="24.894290203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:54.960727545 +0000 UTC m=+28.884272012" watchObservedRunningTime="2026-07-09 18:03:54.960800686 +0000 UTC m=+28.884345175"
Jul 09 18:03:55.934585 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:55.934558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:03:55.935026 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:55.934905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" event={"ID":"ffdf63a8-f00b-4da3-b68c-e45a92e1af50","Type":"ContainerStarted","Data":"6c8e4073c6a28db558777a7c4ddc070708ec54dd98a072ad2cc28a34ac4d4865"}
Jul 09 18:03:55.935218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:55.935178    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:55.935293 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:55.935280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:55.950417 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:55.950393    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:03:55.969642 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:55.969604    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx" podStartSLOduration=8.30071098 podStartE2EDuration="28.969590991s" totalImagesPullingTime="20.668880011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:03:55.9694183 +0000 UTC m=+29.892962772" watchObservedRunningTime="2026-07-09 18:03:55.969590991 +0000 UTC m=+29.893135463"
Jul 09 18:03:56.322999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.322972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m42st"]
Jul 09 18:03:56.323170 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.323090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:56.323232 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:56.323205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:56.326910 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.326849    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7c7ll"]
Jul 09 18:03:56.327038 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.326959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:56.327100 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:56.327059    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:56.327432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.327399    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ghvbn"]
Jul 09 18:03:56.327538 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.327495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:56.327597 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:56.327578    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:56.840711 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.840632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:56.906680 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.906652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:56.938897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.938866    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9f6d09-bbda-4c8d-a4c7-39a062fa1681" containerID="662fc5ff74c39a8648156a420aa425bf49fe0ffc4aeb7b13b7532e2ed73b76dd" exitCode=0
Jul 09 18:03:56.939258 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:56.938943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerDied","Data":"662fc5ff74c39a8648156a420aa425bf49fe0ffc4aeb7b13b7532e2ed73b76dd"}
Jul 09 18:03:57.362564 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.362534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:57.419287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.419258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:57.496018 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.495989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:57.593498 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.593463    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:57.794335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.794256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:57.794335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.794280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:57.794335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.794280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:57.794601 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:57.794382    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:57.794601 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:57.794459    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:57.794601 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:57.794518    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:57.814388 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.814360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:57.914179 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.914141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:57.965062 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:57.965034    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8kwh5_022963e4-a05c-488f-b0c2-b8dbd79305eb/dns-node-resolver/0.log"
Jul 09 18:03:58.093649 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:58.093577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:58.172957 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:58.172917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:58.533726 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:58.533689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:58.592887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:58.592857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:58.945656 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:58.945626    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9f6d09-bbda-4c8d-a4c7-39a062fa1681" containerID="880438bc62eb45eb977ef37e81c3b8a73c5f2bdd382bac3ee11870545bebdaa6" exitCode=0
Jul 09 18:03:58.945801 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:58.945693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerDied","Data":"880438bc62eb45eb977ef37e81c3b8a73c5f2bdd382bac3ee11870545bebdaa6"}
Jul 09 18:03:59.166444 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:59.166422    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wn8fk_6297deef-388a-44dc-8125-5a565307e957/node-ca/0.log"
Jul 09 18:03:59.779130 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:59.779093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:03:59.793812 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:59.793787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:03:59.793952 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:59.793893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:03:59.793952 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:59.793906    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:03:59.793952 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:59.793928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:03:59.794119 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:59.794019    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:03:59.794119 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:03:59.794080    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:03:59.928408 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:03:59.928363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:00.027881 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:00.027849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:00.352956 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:00.352915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:00.352956 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:00.352966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:00.353440 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.353080    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:00.353440 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.353081    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:00.353440 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.353146    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs podName:9cdbabe3-1863-406c-bf94-d8d9933b364e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:32.353127827 +0000 UTC m=+66.276672294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs") pod "network-metrics-daemon-ghvbn" (UID: "9cdbabe3-1863-406c-bf94-d8d9933b364e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:04:00.353440 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.353163    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret podName:c1c19302-1cda-47c9-94aa-ac799575abe2 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:32.353154745 +0000 UTC m=+66.276699199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret") pod "global-pull-secret-syncer-7c7ll" (UID: "c1c19302-1cda-47c9-94aa-ac799575abe2") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:04:00.453827 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:00.453799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:00.454003 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.453935    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:04:00.454003 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.453952    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:04:00.454003 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.453962    2581 projected.go:196] Error preparing data for projected volume kube-api-access-vbvbp for pod openshift-network-diagnostics/network-check-target-m42st: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:00.454162 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:00.454024    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp podName:3298c98f-b3ea-4a84-aa75-ab9b4d233fa1 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:32.454006068 +0000 UTC m=+66.377550522 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vbvbp" (UniqueName: "kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp") pod "network-check-target-m42st" (UID: "3298c98f-b3ea-4a84-aa75-ab9b4d233fa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:04:00.703546 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:00.703456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:01.793568 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:01.793534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:01.793980 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:01.793582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:01.793980 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:01.793601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:01.793980 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:01.793697    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:01.793980 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:01.793732    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:01.793980 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:01.793786    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:03.793811 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:03.793772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:03.793811 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:03.793801    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:03.794464 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:03.793801    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:03.794464 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:03.793913    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:03.794464 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:03.794002    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:03.794464 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:03.794081    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:04.328130 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:04.328032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:04.963604 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:04.963565    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9f6d09-bbda-4c8d-a4c7-39a062fa1681" containerID="24cf03a762b8f43a349a8a8bd7b9e5833034b7ce66cef24b2d70ca4155e270b4" exitCode=0
Jul 09 18:04:04.963966 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:04.963627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerDied","Data":"24cf03a762b8f43a349a8a8bd7b9e5833034b7ce66cef24b2d70ca4155e270b4"}
Jul 09 18:04:05.793596 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.793569    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:05.793596 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.793581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:05.793794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.793581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:05.793794 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:05.793661    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:05.793794 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:05.793739    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:05.793880 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:05.793802    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:05.846260 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.846226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:05.948156 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.948128    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:05.968849 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.968823    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b9f6d09-bbda-4c8d-a4c7-39a062fa1681" containerID="7eea92bd382fa3af5d3fe1af1344cbb8635539f63c80bdf8c02280a8667d0ab9" exitCode=0
Jul 09 18:04:05.969224 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:05.968874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerDied","Data":"7eea92bd382fa3af5d3fe1af1344cbb8635539f63c80bdf8c02280a8667d0ab9"}
Jul 09 18:04:06.856788 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:06.856579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:06.959197 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:06.959161    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:06.974902 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:06.974873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" event={"ID":"1b9f6d09-bbda-4c8d-a4c7-39a062fa1681","Type":"ContainerStarted","Data":"5100bf41b924570066dc5857ad9436f5ea948ff015114dc345d7d8e2868e609c"}
Jul 09 18:04:07.061384 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.061355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:07.164146 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.164069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:07.239004 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.238971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:07.734114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.734086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:07.793783 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.793761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:07.793914 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.793852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:07.793914 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:07.793864    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:07.794011 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:07.793938    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:07.794011 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:07.793986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:07.794127 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:07.794045    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:08.231337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:08.231303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:08.935544 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:08.935514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:09.349833 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.349751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:09.534060 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.534029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:09.741013 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.740984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:09.793978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.793954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:09.794095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.793954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:09.794095 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:09.794076    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:09.794176 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.793954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:09.794176 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:09.794150    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:09.794278 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:09.794229    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:09.938732 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:09.938698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:10.036741 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:10.036664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:10.546324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:10.546290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:10.679592 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:10.679561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.142501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.142470    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.345443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.345410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.454464 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.454370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.554519 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.554487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.633120 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.633077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.750287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.750207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:11.794103 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.794082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:11.794226 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.794082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:11.794226 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:11.794215    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:11.794305 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.794082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:11.794305 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:11.794282    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:11.794369 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:11.794347    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:11.945974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:11.945937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:12.315214 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:12.315150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:12.751793 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:12.751762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:13.006388 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.006316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:13.180633 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.180599    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:13.550952 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.550920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:13.742409 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.742374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:13.794053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.794020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:13.794456 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:13.794118    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:13.794456 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.794125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:13.794456 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:13.794205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:13.794456 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:13.794125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:13.794456 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:13.794271    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:14.141357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:14.141325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:14.543039 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:14.542960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:15.183345 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:15.183307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:15.567271 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:15.567170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:15.793928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:15.793893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:15.793928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:15.793918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:15.794138 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:15.793945    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:15.794138 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:15.794027    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:15.794138 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:15.794078    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:15.794277 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:15.794161    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:15.948390 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:15.948355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:16.153829 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:16.153791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:16.545539 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:16.545500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:17.163831 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:17.163794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:17.350395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:17.350360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:17.538171 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:17.538084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:17.793969 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:17.793886    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:17.793969 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:17.793906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:17.794392 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:17.793887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:17.794392 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:17.794007    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:17.794392 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:17.794055    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:17.794392 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:17.794120    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:19.793500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:19.793463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:19.793500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:19.793495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:19.794067 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:19.793579    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:19.794067 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:19.793641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:19.794067 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:19.793774    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:19.794067 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:19.793861    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:21.771851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:21.771822    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:21.793480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:21.793454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:21.793636 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:21.793547    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:21.793636 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:21.793552    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ghvbn" podUID="9cdbabe3-1863-406c-bf94-d8d9933b364e"
Jul 09 18:04:21.793636 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:21.793573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:21.793752 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:21.793634    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m42st" podUID="3298c98f-b3ea-4a84-aa75-ab9b4d233fa1"
Jul 09 18:04:21.793752 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:21.793693    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7c7ll" podUID="c1c19302-1cda-47c9-94aa-ac799575abe2"
Jul 09 18:04:21.950650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:21.950612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:22.051360 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.051283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:22.401958 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.401934    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-79.ec2.internal" event="NodeReady"
Jul 09 18:04:22.402118 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.402052    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 18:04:22.511789 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.511735    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jpjpd" podStartSLOduration=22.730212815 podStartE2EDuration="55.511718364s" totalImagesPullingTime="32.781505549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:07.004673999 +0000 UTC m=+40.928218468" watchObservedRunningTime="2026-07-09 18:04:22.511718364 +0000 UTC m=+56.435262832"
Jul 09 18:04:22.512095 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.512058    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gc54s"]
Jul 09 18:04:22.528911 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.528878    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f2fxq"]
Jul 09 18:04:22.529074 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.529020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.536324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.536115    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 18:04:22.536324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.536158    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 18:04:22.536324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.536213    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ln8j9\""
Jul 09 18:04:22.543133 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.543110    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f2fxq"]
Jul 09 18:04:22.543268 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.543152    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gc54s"]
Jul 09 18:04:22.543268 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.543238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.546942 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.546905    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 18:04:22.547087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.546975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 18:04:22.547087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.546996    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f5z8w\""
Jul 09 18:04:22.547290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.547228    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 18:04:22.547290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.547262    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 18:04:22.607249 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.607220    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4srdt"]
Jul 09 18:04:22.611737 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-crio-socket\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.611819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-data-volume\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.611819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.611819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83404659-7147-40f3-a6e4-7fb8a477d296-metrics-tls\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.611915 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.611915 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83404659-7147-40f3-a6e4-7fb8a477d296-tmp-dir\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.611976 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bpsk\" (UniqueName: \"kubernetes.io/projected/83404659-7147-40f3-a6e4-7fb8a477d296-kube-api-access-8bpsk\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.611976 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpv62\" (UniqueName: \"kubernetes.io/projected/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-kube-api-access-gpv62\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.612035 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.611990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/83404659-7147-40f3-a6e4-7fb8a477d296-config-volume\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.621347 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.621333    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.624591 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.624567    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4srdt"]
Jul 09 18:04:22.624724 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.624668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 18:04:22.624724 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.624687    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qqkr6\""
Jul 09 18:04:22.625415 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.625398    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 18:04:22.625525 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.625445    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 18:04:22.713227 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83404659-7147-40f3-a6e4-7fb8a477d296-tmp-dir\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.713227 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bpsk\" (UniqueName: \"kubernetes.io/projected/83404659-7147-40f3-a6e4-7fb8a477d296-kube-api-access-8bpsk\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.713227 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpv62\" (UniqueName: \"kubernetes.io/projected/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-kube-api-access-gpv62\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee0ea18-4568-47f0-a889-8f1e623f51ac-cert\") pod \"ingress-canary-4srdt\" (UID: \"5ee0ea18-4568-47f0-a889-8f1e623f51ac\") " pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/83404659-7147-40f3-a6e4-7fb8a477d296-config-volume\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btm26\" (UniqueName: \"kubernetes.io/projected/5ee0ea18-4568-47f0-a889-8f1e623f51ac-kube-api-access-btm26\") pod \"ingress-canary-4srdt\" (UID: \"5ee0ea18-4568-47f0-a889-8f1e623f51ac\") " pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-crio-socket\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-data-volume\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83404659-7147-40f3-a6e4-7fb8a477d296-metrics-tls\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-crio-socket\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713494 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713467    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83404659-7147-40f3-a6e4-7fb8a477d296-tmp-dir\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.713977 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-data-volume\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.713977 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/83404659-7147-40f3-a6e4-7fb8a477d296-config-volume\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.714040 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.713999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.717345 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.717320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.717443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.717322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/83404659-7147-40f3-a6e4-7fb8a477d296-metrics-tls\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.726599 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.726578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpv62\" (UniqueName: \"kubernetes.io/projected/7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5-kube-api-access-gpv62\") pod \"insights-runtime-extractor-f2fxq\" (UID: \"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5\") " pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.729350 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.729321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bpsk\" (UniqueName: \"kubernetes.io/projected/83404659-7147-40f3-a6e4-7fb8a477d296-kube-api-access-8bpsk\") pod \"dns-default-gc54s\" (UID: \"83404659-7147-40f3-a6e4-7fb8a477d296\") " pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.814224 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.814181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee0ea18-4568-47f0-a889-8f1e623f51ac-cert\") pod \"ingress-canary-4srdt\" (UID: \"5ee0ea18-4568-47f0-a889-8f1e623f51ac\") " pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.814224 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.814235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btm26\" (UniqueName: \"kubernetes.io/projected/5ee0ea18-4568-47f0-a889-8f1e623f51ac-kube-api-access-btm26\") pod \"ingress-canary-4srdt\" (UID: \"5ee0ea18-4568-47f0-a889-8f1e623f51ac\") " pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.816508 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.816481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee0ea18-4568-47f0-a889-8f1e623f51ac-cert\") pod \"ingress-canary-4srdt\" (UID: \"5ee0ea18-4568-47f0-a889-8f1e623f51ac\") " pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.825242 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.825224    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btm26\" (UniqueName: \"kubernetes.io/projected/5ee0ea18-4568-47f0-a889-8f1e623f51ac-kube-api-access-btm26\") pod \"ingress-canary-4srdt\" (UID: \"5ee0ea18-4568-47f0-a889-8f1e623f51ac\") " pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:22.850274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.850245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:22.851321 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.851307    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f2fxq"
Jul 09 18:04:22.931306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:22.931269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4srdt"
Jul 09 18:04:23.090594 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.090564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f2fxq"]
Jul 09 18:04:23.093937 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:23.093906    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c1d1a14_13fc_4fa7_a6dd_dbbedd9b18e5.slice/crio-0dae890c3ddd6b6e820f8157df6d62724bf94da31b0909c495fdb4789ac91793 WatchSource:0}: Error finding container 0dae890c3ddd6b6e820f8157df6d62724bf94da31b0909c495fdb4789ac91793: Status 404 returned error can't find the container with id 0dae890c3ddd6b6e820f8157df6d62724bf94da31b0909c495fdb4789ac91793
Jul 09 18:04:23.094264 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.094247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gc54s"]
Jul 09 18:04:23.097433 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:23.097409    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83404659_7147_40f3_a6e4_7fb8a477d296.slice/crio-5e52d0371eecf0a48604f09e27e9c749b4933a811fbdac5f9dbb12f65492e6eb WatchSource:0}: Error finding container 5e52d0371eecf0a48604f09e27e9c749b4933a811fbdac5f9dbb12f65492e6eb: Status 404 returned error can't find the container with id 5e52d0371eecf0a48604f09e27e9c749b4933a811fbdac5f9dbb12f65492e6eb
Jul 09 18:04:23.101081 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.101056    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:23.108772 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.108680    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4srdt"]
Jul 09 18:04:23.111850 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:23.111827    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee0ea18_4568_47f0_a889_8f1e623f51ac.slice/crio-4fbe74471b59f474481489219d2c313ac4155e25836846856aae21ca82fa594a WatchSource:0}: Error finding container 4fbe74471b59f474481489219d2c313ac4155e25836846856aae21ca82fa594a: Status 404 returned error can't find the container with id 4fbe74471b59f474481489219d2c313ac4155e25836846856aae21ca82fa594a
Jul 09 18:04:23.179554 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.179516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:23.794320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.794282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:23.794747 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.794729    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:23.795202 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.795169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:23.797871 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.797849    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:04:23.797975 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.797954    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 18:04:23.799279 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.799259    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cmkm4\""
Jul 09 18:04:23.799544 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.799526    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:04:23.799756 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.799739    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hzqhq\""
Jul 09 18:04:23.800019 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:23.800001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 18:04:24.017888 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:24.017850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4srdt" event={"ID":"5ee0ea18-4568-47f0-a889-8f1e623f51ac","Type":"ContainerStarted","Data":"4fbe74471b59f474481489219d2c313ac4155e25836846856aae21ca82fa594a"}
Jul 09 18:04:24.020941 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:24.020909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2fxq" event={"ID":"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5","Type":"ContainerStarted","Data":"0dae890c3ddd6b6e820f8157df6d62724bf94da31b0909c495fdb4789ac91793"}
Jul 09 18:04:24.023876 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:24.023848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gc54s" event={"ID":"83404659-7147-40f3-a6e4-7fb8a477d296","Type":"ContainerStarted","Data":"5e52d0371eecf0a48604f09e27e9c749b4933a811fbdac5f9dbb12f65492e6eb"}
Jul 09 18:04:25.618764 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.618725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:25.719932 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.719898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:25.788876 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.788843    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"]
Jul 09 18:04:25.794002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.793975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:25.796618 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.796591    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kdhm4\""
Jul 09 18:04:25.796753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.796690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 18:04:25.804570 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.804549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"]
Jul 09 18:04:25.813257 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.813229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:25.839905 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.839873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/099445ac-a094-476d-94c2-a109fb2a053e-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fnqrs\" (UID: \"099445ac-a094-476d-94c2-a109fb2a053e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:25.940704 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.940626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/099445ac-a094-476d-94c2-a109fb2a053e-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fnqrs\" (UID: \"099445ac-a094-476d-94c2-a109fb2a053e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:25.943425 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:25.943369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/099445ac-a094-476d-94c2-a109fb2a053e-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-fnqrs\" (UID: \"099445ac-a094-476d-94c2-a109fb2a053e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:26.061595 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:26.061559    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:26.104335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:26.104299    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:26.181692 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:26.181652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:26.280973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:26.280897    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:27.279046 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:27.278995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:27.359977 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:27.359932    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:27.952525 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:27.952492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hvbkx"
Jul 09 18:04:28.153871 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:28.153771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:28.255441 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:28.255390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:28.329244 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:28.327774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"]
Jul 09 18:04:28.333454 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:28.333421    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099445ac_a094_476d_94c2_a109fb2a053e.slice/crio-a9a4cc50220c6ab33711ee6254d05b60b9da40919cb1dbb05d3983e79bcd517a WatchSource:0}: Error finding container a9a4cc50220c6ab33711ee6254d05b60b9da40919cb1dbb05d3983e79bcd517a: Status 404 returned error can't find the container with id a9a4cc50220c6ab33711ee6254d05b60b9da40919cb1dbb05d3983e79bcd517a
Jul 09 18:04:28.642175 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:28.642135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:28.749750 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:28.749720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:29.041376 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.041344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gc54s" event={"ID":"83404659-7147-40f3-a6e4-7fb8a477d296","Type":"ContainerStarted","Data":"25ab139c14b524fe9f0c64f54e02517d398ddc99eb24c67963104e6064a07d45"}
Jul 09 18:04:29.041543 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.041401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gc54s" event={"ID":"83404659-7147-40f3-a6e4-7fb8a477d296","Type":"ContainerStarted","Data":"eb9f86afe8df7a632d05ffa4166bc80df1847c6e5ef67a5be928e93c7cb96937"}
Jul 09 18:04:29.041543 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.041431    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:29.042630 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.042602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs" event={"ID":"099445ac-a094-476d-94c2-a109fb2a053e","Type":"ContainerStarted","Data":"a9a4cc50220c6ab33711ee6254d05b60b9da40919cb1dbb05d3983e79bcd517a"}
Jul 09 18:04:29.044142 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.044122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4srdt" event={"ID":"5ee0ea18-4568-47f0-a889-8f1e623f51ac","Type":"ContainerStarted","Data":"4f66891b92b02d512c19aaeb5dbf55a7a2968a8d6280aaa8050da92ce595ed35"}
Jul 09 18:04:29.045940 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.045920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2fxq" event={"ID":"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5","Type":"ContainerStarted","Data":"c84fbe22a72075c2218513113105d964d7321df65214d39871a36a72e44956d7"}
Jul 09 18:04:29.046025 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.045967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2fxq" event={"ID":"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5","Type":"ContainerStarted","Data":"f6473ab233e2c4e3f9d927c950c33b2715cc948ff2a88d7847bb0856539fcd08"}
Jul 09 18:04:29.065127 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.065086    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gc54s" podStartSLOduration=1.975760104 podStartE2EDuration="7.065073134s" totalImagesPullingTime="5.08931303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:29.064778109 +0000 UTC m=+62.988322581" watchObservedRunningTime="2026-07-09 18:04:29.065073134 +0000 UTC m=+62.988617605"
Jul 09 18:04:29.084094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:29.084042    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4srdt" podStartSLOduration=2.008952462 podStartE2EDuration="7.084027661s" totalImagesPullingTime="5.075075199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:29.083611722 +0000 UTC m=+63.007156190" watchObservedRunningTime="2026-07-09 18:04:29.084027661 +0000 UTC m=+63.007572133"
Jul 09 18:04:31.055911 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.055872    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs" event={"ID":"099445ac-a094-476d-94c2-a109fb2a053e","Type":"ContainerStarted","Data":"51d6dfaeae5640566b774853a2b949939a161c1652a5bb14ca8cdf1bf1b2eb82"}
Jul 09 18:04:31.056384 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.056040    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:31.062215 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.062176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs"
Jul 09 18:04:31.072728 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.072683    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-fnqrs" podStartSLOduration=4.248890001 podStartE2EDuration="6.072668332s" totalImagesPullingTime="1.823778331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:31.071774523 +0000 UTC m=+64.995318997" watchObservedRunningTime="2026-07-09 18:04:31.072668332 +0000 UTC m=+64.996212828"
Jul 09 18:04:31.355994 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.355959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:31.428442 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.428400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:31.860092 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.860057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:31.893062 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.893036    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-kmfk8"]
Jul 09 18:04:31.894876 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.894861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:31.898516 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.898490    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t58mt\""
Jul 09 18:04:31.898626 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.898491    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 18:04:31.898626 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.898492    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 18:04:31.898626 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.898564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 18:04:31.898626 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.898491    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 18:04:31.898834 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.898541    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 18:04:31.908755 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.908737    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-kmfk8"]
Jul 09 18:04:31.928231 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.928180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:31.989343 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.989315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c85b356f-d322-47de-aaef-946709cb5e5a-metrics-client-ca\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:31.989470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.989350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c85b356f-d322-47de-aaef-946709cb5e5a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:31.989470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.989378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85b356f-d322-47de-aaef-946709cb5e5a-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:31.989470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.989417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vrdg\" (UniqueName: \"kubernetes.io/projected/c85b356f-d322-47de-aaef-946709cb5e5a-kube-api-access-6vrdg\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:31.999335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:31.999293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:32.060568 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.060530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2fxq" event={"ID":"7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5","Type":"ContainerStarted","Data":"2a1d3183476fe3c1a20be455f9ad32f4116a6d8a40acb47fd90ba1100b97b03a"}
Jul 09 18:04:32.081112 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.081064    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f2fxq" podStartSLOduration=1.868389032 podStartE2EDuration="10.081045835s" totalImagesPullingTime="8.212656803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:32.08054848 +0000 UTC m=+66.004092951" watchObservedRunningTime="2026-07-09 18:04:32.081045835 +0000 UTC m=+66.004590286"
Jul 09 18:04:32.089716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.089696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85b356f-d322-47de-aaef-946709cb5e5a-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.089793 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.089743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vrdg\" (UniqueName: \"kubernetes.io/projected/c85b356f-d322-47de-aaef-946709cb5e5a-kube-api-access-6vrdg\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.089950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.089932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c85b356f-d322-47de-aaef-946709cb5e5a-metrics-client-ca\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.090017 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.089970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c85b356f-d322-47de-aaef-946709cb5e5a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.090630 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.090610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c85b356f-d322-47de-aaef-946709cb5e5a-metrics-client-ca\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.092118 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.092098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85b356f-d322-47de-aaef-946709cb5e5a-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.092209 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.092102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c85b356f-d322-47de-aaef-946709cb5e5a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.097915 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.097893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vrdg\" (UniqueName: \"kubernetes.io/projected/c85b356f-d322-47de-aaef-946709cb5e5a-kube-api-access-6vrdg\") pod \"prometheus-operator-64785b6675-kmfk8\" (UID: \"c85b356f-d322-47de-aaef-946709cb5e5a\") " pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.203122 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.203027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8"
Jul 09 18:04:32.333614 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.333584    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-kmfk8"]
Jul 09 18:04:32.336875 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:32.336851    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85b356f_d322_47de_aaef_946709cb5e5a.slice/crio-8aab3a01f0d1fba22123b7c94b4c625b0ac9a50206f36339415aff8e2cb46b48 WatchSource:0}: Error finding container 8aab3a01f0d1fba22123b7c94b4c625b0ac9a50206f36339415aff8e2cb46b48: Status 404 returned error can't find the container with id 8aab3a01f0d1fba22123b7c94b4c625b0ac9a50206f36339415aff8e2cb46b48
Jul 09 18:04:32.392620 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.392586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:32.392779 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.392627    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:32.395036 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.395017    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:04:32.395121 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.395054    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:04:32.405304 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.405276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cdbabe3-1863-406c-bf94-d8d9933b364e-metrics-certs\") pod \"network-metrics-daemon-ghvbn\" (UID: \"9cdbabe3-1863-406c-bf94-d8d9933b364e\") " pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:32.405304 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.405297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1c19302-1cda-47c9-94aa-ac799575abe2-original-pull-secret\") pod \"global-pull-secret-syncer-7c7ll\" (UID: \"c1c19302-1cda-47c9-94aa-ac799575abe2\") " pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:32.493562 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.493489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:32.496516 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.496497    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 18:04:32.506994 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.506969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 18:04:32.513427 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.513407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cmkm4\""
Jul 09 18:04:32.517705 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.517684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbvbp\" (UniqueName: \"kubernetes.io/projected/3298c98f-b3ea-4a84-aa75-ab9b4d233fa1-kube-api-access-vbvbp\") pod \"network-check-target-m42st\" (UID: \"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1\") " pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:32.521480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.521451    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ghvbn"
Jul 09 18:04:32.523044 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.523028    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hzqhq\""
Jul 09 18:04:32.529377 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.529354    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7c7ll"
Jul 09 18:04:32.530938 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.530894    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:32.672455 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.672425    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ghvbn"]
Jul 09 18:04:32.675444 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:32.675408    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cdbabe3_1863_406c_bf94_d8d9933b364e.slice/crio-ec717b1425702fafd53ac33215607b5ea573f26f9ffcf779b6008663aaeccd60 WatchSource:0}: Error finding container ec717b1425702fafd53ac33215607b5ea573f26f9ffcf779b6008663aaeccd60: Status 404 returned error can't find the container with id ec717b1425702fafd53ac33215607b5ea573f26f9ffcf779b6008663aaeccd60
Jul 09 18:04:32.891126 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.891084    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7c7ll"]
Jul 09 18:04:32.891978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:32.891957    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m42st"]
Jul 09 18:04:32.894698 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:32.894672    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3298c98f_b3ea_4a84_aa75_ab9b4d233fa1.slice/crio-0349713f43ff031d7d39b83a351638cb26feb9bdeabb3000ae34c2ad6d89011c WatchSource:0}: Error finding container 0349713f43ff031d7d39b83a351638cb26feb9bdeabb3000ae34c2ad6d89011c: Status 404 returned error can't find the container with id 0349713f43ff031d7d39b83a351638cb26feb9bdeabb3000ae34c2ad6d89011c
Jul 09 18:04:32.896406 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:32.896242    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c19302_1cda_47c9_94aa_ac799575abe2.slice/crio-cabdd674996481acd5c908ecd918995da6476007e02b9d81685a00877990888d WatchSource:0}: Error finding container cabdd674996481acd5c908ecd918995da6476007e02b9d81685a00877990888d: Status 404 returned error can't find the container with id cabdd674996481acd5c908ecd918995da6476007e02b9d81685a00877990888d
Jul 09 18:04:33.004165 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.004125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:33.065022 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.064990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m42st" event={"ID":"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1","Type":"ContainerStarted","Data":"0349713f43ff031d7d39b83a351638cb26feb9bdeabb3000ae34c2ad6d89011c"}
Jul 09 18:04:33.066080 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.066053    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ghvbn" event={"ID":"9cdbabe3-1863-406c-bf94-d8d9933b364e","Type":"ContainerStarted","Data":"ec717b1425702fafd53ac33215607b5ea573f26f9ffcf779b6008663aaeccd60"}
Jul 09 18:04:33.067147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.067128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8" event={"ID":"c85b356f-d322-47de-aaef-946709cb5e5a","Type":"ContainerStarted","Data":"8aab3a01f0d1fba22123b7c94b4c625b0ac9a50206f36339415aff8e2cb46b48"}
Jul 09 18:04:33.068148 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.068125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7c7ll" event={"ID":"c1c19302-1cda-47c9-94aa-ac799575abe2","Type":"ContainerStarted","Data":"cabdd674996481acd5c908ecd918995da6476007e02b9d81685a00877990888d"}
Jul 09 18:04:33.129418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.129379    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:33.360069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:33.360038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:34.005742 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:34.005645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:34.075145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:34.075085    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8" event={"ID":"c85b356f-d322-47de-aaef-946709cb5e5a","Type":"ContainerStarted","Data":"6855d9cc4e87a1a9e370ab0e69efe3e504bb6f8755c59e6d37c1ff3fdf80acbd"}
Jul 09 18:04:34.075611 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:34.075153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8" event={"ID":"c85b356f-d322-47de-aaef-946709cb5e5a","Type":"ContainerStarted","Data":"bc5d6c75e01c16209b5dff9baa6c7ee0f47b658d13eaa605905763784325688e"}
Jul 09 18:04:34.105403 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:34.105342    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-kmfk8" podStartSLOduration=1.739731497 podStartE2EDuration="3.105319128s" totalImagesPullingTime="1.365587631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:34.102346541 +0000 UTC m=+68.025891006" watchObservedRunningTime="2026-07-09 18:04:34.105319128 +0000 UTC m=+68.028863601"
Jul 09 18:04:35.081806 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.081768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ghvbn" event={"ID":"9cdbabe3-1863-406c-bf94-d8d9933b364e","Type":"ContainerStarted","Data":"012f7c1408ac5707c434624f8cb6772d94cde08b1144bd6684bd5e3d626ffde1"}
Jul 09 18:04:35.082286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.081829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ghvbn" event={"ID":"9cdbabe3-1863-406c-bf94-d8d9933b364e","Type":"ContainerStarted","Data":"1f40a2285803c2d25b23b4e67cdec1c15575bca3748af423ebe9b8232c513fcc"}
Jul 09 18:04:35.101144 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.100512    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ghvbn" podStartSLOduration=66.322681405 podStartE2EDuration="1m8.100490058s" totalImagesPullingTime="1.777808653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:35.099337294 +0000 UTC m=+69.022881764" watchObservedRunningTime="2026-07-09 18:04:35.100490058 +0000 UTC m=+69.024034533"
Jul 09 18:04:35.224702 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.224664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:35.287328 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.287289    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-f54r7"]
Jul 09 18:04:35.289477 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.289449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:35.292177 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.292155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zt4kh\""
Jul 09 18:04:35.292559 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.292537    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 18:04:35.292671 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.292652    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 18:04:35.304538 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.304516    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-f54r7"]
Jul 09 18:04:35.339316 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.339231    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:35.414320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.414283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7glfr\" (UniqueName: \"kubernetes.io/projected/6d16fd67-63e1-4a66-8088-5f8c62bd72cb-kube-api-access-7glfr\") pod \"downloads-68c9cbb497-f54r7\" (UID: \"6d16fd67-63e1-4a66-8088-5f8c62bd72cb\") " pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:35.435322 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.435288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:35.515570 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.515533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7glfr\" (UniqueName: \"kubernetes.io/projected/6d16fd67-63e1-4a66-8088-5f8c62bd72cb-kube-api-access-7glfr\") pod \"downloads-68c9cbb497-f54r7\" (UID: \"6d16fd67-63e1-4a66-8088-5f8c62bd72cb\") " pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:35.525066 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.525039    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7glfr\" (UniqueName: \"kubernetes.io/projected/6d16fd67-63e1-4a66-8088-5f8c62bd72cb-kube-api-access-7glfr\") pod \"downloads-68c9cbb497-f54r7\" (UID: \"6d16fd67-63e1-4a66-8088-5f8c62bd72cb\") " pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:35.577307 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.577274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:35.601256 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.601231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:35.745992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.744276    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-f54r7"]
Jul 09 18:04:35.752022 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.751997    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:35.824697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:35.824659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.061263 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.061156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.213422 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.213387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.317214 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.317112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.421815 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.421787    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"]
Jul 09 18:04:36.432784 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.432755    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.456259 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.456234    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"]
Jul 09 18:04:36.456403 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.456292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.460251 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.460227    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 18:04:36.460372 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.460256    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hhbgj\""
Jul 09 18:04:36.460525 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.460507    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 18:04:36.464811 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.464793    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-cmr54"]
Jul 09 18:04:36.482077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.482045    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.483578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.483551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-cmr54"]
Jul 09 18:04:36.485094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.485074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 18:04:36.485210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.485126    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 18:04:36.485460 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.485439    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 18:04:36.486099 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.486044    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cnlg2\""
Jul 09 18:04:36.524961 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.524931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3255187c-5222-4fbb-89a7-0507b85ea06e-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.525114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.524972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.525114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.525114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.525114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl5n4\" (UniqueName: \"kubernetes.io/projected/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-kube-api-access-tl5n4\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.525323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.525323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvn9p\" (UniqueName: \"kubernetes.io/projected/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-api-access-gvn9p\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.525323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.525323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.525323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.525245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3255187c-5222-4fbb-89a7-0507b85ea06e-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.530062 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.530037    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8tnlx"]
Jul 09 18:04:36.552267 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.552240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.554895 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.554877    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 18:04:36.554996 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.554881    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ztzgw\""
Jul 09 18:04:36.555158 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.555141    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 18:04:36.555998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.555979    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 18:04:36.605683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.605654    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.625870 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.625847    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-root\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.625999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.625880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-textfile\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.625999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.625913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.625999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.625979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.625999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.625998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz485\" (UniqueName: \"kubernetes.io/projected/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-kube-api-access-wz485\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3255187c-5222-4fbb-89a7-0507b85ea06e-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.626212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-sys\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-wtmp\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3255187c-5222-4fbb-89a7-0507b85ea06e-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.626212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.626471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-accelerators-collector-config\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-metrics-client-ca\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-tls\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.626471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.626697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.626697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3255187c-5222-4fbb-89a7-0507b85ea06e-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.626697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl5n4\" (UniqueName: \"kubernetes.io/projected/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-kube-api-access-tl5n4\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.626900 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.626989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.626920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvn9p\" (UniqueName: \"kubernetes.io/projected/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-api-access-gvn9p\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.627044 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.627015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3255187c-5222-4fbb-89a7-0507b85ea06e-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.627097 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.627015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.627583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.627559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.629046 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.628986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.629605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.629582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.630002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.629961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.630269 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.630245    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.641605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.641579    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl5n4\" (UniqueName: \"kubernetes.io/projected/6fb29d9d-a69b-4ba4-92a5-dacad1c693aa-kube-api-access-tl5n4\") pod \"openshift-state-metrics-84bd68b896-kpdm5\" (UID: \"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.642004 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.641986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvn9p\" (UniqueName: \"kubernetes.io/projected/3255187c-5222-4fbb-89a7-0507b85ea06e-kube-api-access-gvn9p\") pod \"kube-state-metrics-785778ff74-cmr54\" (UID: \"3255187c-5222-4fbb-89a7-0507b85ea06e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.716588 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.716540    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.727529 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-root\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727529 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-textfile\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz485\" (UniqueName: \"kubernetes.io/projected/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-kube-api-access-wz485\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-sys\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-wtmp\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727639    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-root\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-accelerators-collector-config\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.727716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-metrics-client-ca\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728006 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727738    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-sys\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728006 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728006 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.727796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-tls\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728006 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:36.727944    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 18:04:36.728243 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:04:36.728016    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-tls podName:2e2874ca-13c1-432b-b37c-2ebe4fc1fc03 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:37.227992276 +0000 UTC m=+71.151536728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-tls") pod "node-exporter-8tnlx" (UID: "2e2874ca-13c1-432b-b37c-2ebe4fc1fc03") : secret "node-exporter-tls" not found
Jul 09 18:04:36.728243 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.728179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-wtmp\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728243 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.728233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-textfile\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728406 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.728269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-accelerators-collector-config\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.728630 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.728610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-metrics-client-ca\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.730515 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.730480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.747229 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.747180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz485\" (UniqueName: \"kubernetes.io/projected/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-kube-api-access-wz485\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:36.768009 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.767988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"
Jul 09 18:04:36.793275 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.793255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54"
Jul 09 18:04:36.823883 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.823850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.846770 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:36.846737    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d16fd67_63e1_4a66_8088_5f8c62bd72cb.slice/crio-53b3c839ef568278586dbe817396d7d56cdc40c947fe638b9002ddef07b2acb9 WatchSource:0}: Error finding container 53b3c839ef568278586dbe817396d7d56cdc40c947fe638b9002ddef07b2acb9: Status 404 returned error can't find the container with id 53b3c839ef568278586dbe817396d7d56cdc40c947fe638b9002ddef07b2acb9
Jul 09 18:04:36.963905 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.963857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:36.995418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:36.993140    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-cmr54"]
Jul 09 18:04:36.998947 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:36.998919    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3255187c_5222_4fbb_89a7_0507b85ea06e.slice/crio-fa51460e05197b9c9d1f1c550d5c23b3a5ed8e1808fe084f844b26468a499cef WatchSource:0}: Error finding container fa51460e05197b9c9d1f1c550d5c23b3a5ed8e1808fe084f844b26468a499cef: Status 404 returned error can't find the container with id fa51460e05197b9c9d1f1c550d5c23b3a5ed8e1808fe084f844b26468a499cef
Jul 09 18:04:37.006757 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.006735    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5"]
Jul 09 18:04:37.009759 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:37.009737    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb29d9d_a69b_4ba4_92a5_dacad1c693aa.slice/crio-5409f8ac83a28982cbb0a674ceb15bdae684c703e4a87adccc616173ec4f82cc WatchSource:0}: Error finding container 5409f8ac83a28982cbb0a674ceb15bdae684c703e4a87adccc616173ec4f82cc: Status 404 returned error can't find the container with id 5409f8ac83a28982cbb0a674ceb15bdae684c703e4a87adccc616173ec4f82cc
Jul 09 18:04:37.092207 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.092152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54" event={"ID":"3255187c-5222-4fbb-89a7-0507b85ea06e","Type":"ContainerStarted","Data":"fa51460e05197b9c9d1f1c550d5c23b3a5ed8e1808fe084f844b26468a499cef"}
Jul 09 18:04:37.093366 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.093338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-f54r7" event={"ID":"6d16fd67-63e1-4a66-8088-5f8c62bd72cb","Type":"ContainerStarted","Data":"53b3c839ef568278586dbe817396d7d56cdc40c947fe638b9002ddef07b2acb9"}
Jul 09 18:04:37.094549 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.094527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5" event={"ID":"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa","Type":"ContainerStarted","Data":"5409f8ac83a28982cbb0a674ceb15bdae684c703e4a87adccc616173ec4f82cc"}
Jul 09 18:04:37.219170 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.219091    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:37.231113 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.231080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-tls\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:37.233393 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.233372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e2874ca-13c1-432b-b37c-2ebe4fc1fc03-node-exporter-tls\") pod \"node-exporter-8tnlx\" (UID: \"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03\") " pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:37.332495 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.332359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:37.463544 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.463518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8tnlx"
Jul 09 18:04:37.463827 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.463797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:37.490780 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:37.490746    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2874ca_13c1_432b_b37c_2ebe4fc1fc03.slice/crio-478fefbd9045d7f6f26c58837e08d42125584d565f56ffb747174647ea35b581 WatchSource:0}: Error finding container 478fefbd9045d7f6f26c58837e08d42125584d565f56ffb747174647ea35b581: Status 404 returned error can't find the container with id 478fefbd9045d7f6f26c58837e08d42125584d565f56ffb747174647ea35b581
Jul 09 18:04:37.601638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.601572    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:37.652623 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.652595    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:04:37.667890 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.667855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.670771 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.670737    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 18:04:37.670944 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.670929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 18:04:37.671129 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671115    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 18:04:37.671729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671393    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 18:04:37.671729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671472    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 18:04:37.671729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671495    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 18:04:37.671729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671502    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 18:04:37.671729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671596    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 18:04:37.671729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.671624    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dlv55\""
Jul 09 18:04:37.681953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.681173    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 18:04:37.682117 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.682095    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:04:37.710080 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.710048    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:37.734872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.734872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.734872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-web-config\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-config-volume\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734970    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvnhv\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-kube-api-access-pvnhv\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.734991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.735011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-config-out\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.735028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.735051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.735073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.735114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.735092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.835864 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.835864 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-config-out\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835888    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.835992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.836046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836088 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.836078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836421 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.836101    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-web-config\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836421 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.836157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-config-volume\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836421 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.836209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.836421 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.836238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvnhv\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-kube-api-access-pvnhv\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.837384 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.837357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.838410 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.838378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.838619 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.838596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.842697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.842561    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.842697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.842564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.842697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.842653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.842997 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.842957    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-config-volume\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.842997 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.842977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.843121 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.843060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-config-out\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.843451 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.843408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.843937 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.843913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.844338 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.844316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-web-config\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.866730 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.866675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvnhv\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-kube-api-access-pvnhv\") pod \"alertmanager-main-0\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:37.912418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.912249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:37.985034 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:37.984951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:04:38.046199 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.046028    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:38.108092 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.107947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5" event={"ID":"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa","Type":"ContainerStarted","Data":"272acbaafb6074aaa8fe0cab2bfb53a6c385844a3c07c99512c3f787fa8331c2"}
Jul 09 18:04:38.108092 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.108017    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5" event={"ID":"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa","Type":"ContainerStarted","Data":"c30ab3dbb01a18d65873fe92f8070d770bb9294a1626ed8f733f32e4c700d4f2"}
Jul 09 18:04:38.114200 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.113411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m42st" event={"ID":"3298c98f-b3ea-4a84-aa75-ab9b4d233fa1","Type":"ContainerStarted","Data":"b3987d249ee1149ff11766d4e1f3934a5741558e85dbb73300edd62c67397d0e"}
Jul 09 18:04:38.114200 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.114125    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:04:38.116409 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.116355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8tnlx" event={"ID":"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03","Type":"ContainerStarted","Data":"478fefbd9045d7f6f26c58837e08d42125584d565f56ffb747174647ea35b581"}
Jul 09 18:04:38.202162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.201242    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m42st" podStartSLOduration=66.838061976 podStartE2EDuration="1m11.201218953s" totalImagesPullingTime="4.363156977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:38.146117453 +0000 UTC m=+72.069661928" watchObservedRunningTime="2026-07-09 18:04:38.201218953 +0000 UTC m=+72.124763434"
Jul 09 18:04:38.202162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:38.201966    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:04:39.054154 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.054120    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gc54s"
Jul 09 18:04:39.125411 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.125372    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"fd6da4778c75af47d61732f939f2130bd6261f08c31461fa2083904d6ff5f26d"}
Jul 09 18:04:39.271288 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.271238    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:39.374922 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.374841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:39.478345 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.478292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:39.534223 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.534171    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7cb5558858-grx86"]
Jul 09 18:04:39.537834 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.537813    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.540744 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.540719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 18:04:39.540850 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.540758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-84fvv\""
Jul 09 18:04:39.540850 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.540777    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-al7q5528s50rb\""
Jul 09 18:04:39.541092 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.541069    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 18:04:39.541172 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.541120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 18:04:39.541394 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.541373    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 18:04:39.541492 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.541420    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 18:04:39.553578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.553550    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7cb5558858-grx86"]
Jul 09 18:04:39.570249 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.570221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:39.653723 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.653723 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.653723 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-grpc-tls\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.653723 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2t29\" (UniqueName: \"kubernetes.io/projected/a31a3680-a10f-43bc-a288-54b6bc6f8035-kube-api-access-j2t29\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.654011 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.654011 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653823    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31a3680-a10f-43bc-a288-54b6bc6f8035-metrics-client-ca\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.654011 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.654011 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.653917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-tls\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.754817 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.754780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31a3680-a10f-43bc-a288-54b6bc6f8035-metrics-client-ca\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.754835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.754879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-tls\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.754917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.754948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.754982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-grpc-tls\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.755008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2t29\" (UniqueName: \"kubernetes.io/projected/a31a3680-a10f-43bc-a288-54b6bc6f8035-kube-api-access-j2t29\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.755060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.755819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.755688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a31a3680-a10f-43bc-a288-54b6bc6f8035-metrics-client-ca\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.758687 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.758622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.758929 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.758905    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.759096 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.759073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-grpc-tls\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.759982 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.759936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.760278 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.760256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-tls\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.760654 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.760631    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a31a3680-a10f-43bc-a288-54b6bc6f8035-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.766484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.766441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2t29\" (UniqueName: \"kubernetes.io/projected/a31a3680-a10f-43bc-a288-54b6bc6f8035-kube-api-access-j2t29\") pod \"thanos-querier-7cb5558858-grx86\" (UID: \"a31a3680-a10f-43bc-a288-54b6bc6f8035\") " pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.849606 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.849571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:39.887573 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:39.887415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:40.507886 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.507851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:40.862005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.861970    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fb696657c-dww2m"]
Jul 09 18:04:40.865779 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.865750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.872152 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.872049    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 18:04:40.873341 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.873157    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jvf8s\""
Jul 09 18:04:40.873341 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.873300    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 18:04:40.873341 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.873317    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 18:04:40.873341 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.873323    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d9kaf71htknhn\""
Jul 09 18:04:40.873611 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.873343    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 18:04:40.889266 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.889234    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fb696657c-dww2m"]
Jul 09 18:04:40.967908 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.967824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/235c39b9-9046-4a1d-abbb-210284c39933-metrics-server-audit-profiles\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.967908 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.967889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/235c39b9-9046-4a1d-abbb-210284c39933-audit-log\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.968219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.967931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/235c39b9-9046-4a1d-abbb-210284c39933-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.968219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.967960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-secret-metrics-server-tls\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.968219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.968073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-client-ca-bundle\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.968219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.968127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stp8n\" (UniqueName: \"kubernetes.io/projected/235c39b9-9046-4a1d-abbb-210284c39933-kube-api-access-stp8n\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:40.968219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:40.968165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-secret-metrics-server-client-certs\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-client-ca-bundle\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stp8n\" (UniqueName: \"kubernetes.io/projected/235c39b9-9046-4a1d-abbb-210284c39933-kube-api-access-stp8n\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-secret-metrics-server-client-certs\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/235c39b9-9046-4a1d-abbb-210284c39933-metrics-server-audit-profiles\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/235c39b9-9046-4a1d-abbb-210284c39933-audit-log\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069776    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/235c39b9-9046-4a1d-abbb-210284c39933-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.070247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.069804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-secret-metrics-server-tls\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.071438 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.071058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/235c39b9-9046-4a1d-abbb-210284c39933-audit-log\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.071438 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.071056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/235c39b9-9046-4a1d-abbb-210284c39933-metrics-server-audit-profiles\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.071438 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.071426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/235c39b9-9046-4a1d-abbb-210284c39933-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.072959 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.072908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-client-ca-bundle\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.074122 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.074097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-secret-metrics-server-client-certs\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.074215 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.074142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/235c39b9-9046-4a1d-abbb-210284c39933-secret-metrics-server-tls\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.101087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.101026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stp8n\" (UniqueName: \"kubernetes.io/projected/235c39b9-9046-4a1d-abbb-210284c39933-kube-api-access-stp8n\") pod \"metrics-server-6fb696657c-dww2m\" (UID: \"235c39b9-9046-4a1d-abbb-210284c39933\") " pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.178314 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.178278    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7cb5558858-grx86"]
Jul 09 18:04:41.178534 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.178494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:04:41.287246 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.287171    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"]
Jul 09 18:04:41.290749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.290686    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.294591 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.294567    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 09 18:04:41.294864 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.294845    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-s87jd\""
Jul 09 18:04:41.336073 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.336002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"]
Jul 09 18:04:41.354377 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:41.354269    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31a3680_a10f_43bc_a288_54b6bc6f8035.slice/crio-5bf1842e5d47949e51d3513206947de40a2fe3dc11d15d85cc5c91be8cc00074 WatchSource:0}: Error finding container 5bf1842e5d47949e51d3513206947de40a2fe3dc11d15d85cc5c91be8cc00074: Status 404 returned error can't find the container with id 5bf1842e5d47949e51d3513206947de40a2fe3dc11d15d85cc5c91be8cc00074
Jul 09 18:04:41.373334 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.373300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8868369b-4bd8-4d79-abcb-4ac42db6fa0f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-plj97\" (UID: \"8868369b-4bd8-4d79-abcb-4ac42db6fa0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.373477 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.373344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-826kl\" (UniqueName: \"kubernetes.io/projected/8868369b-4bd8-4d79-abcb-4ac42db6fa0f-kube-api-access-826kl\") pod \"monitoring-plugin-5f68df7f58-plj97\" (UID: \"8868369b-4bd8-4d79-abcb-4ac42db6fa0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.383505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.383359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:41.475542 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.474579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8868369b-4bd8-4d79-abcb-4ac42db6fa0f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-plj97\" (UID: \"8868369b-4bd8-4d79-abcb-4ac42db6fa0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.475542 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.474641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-826kl\" (UniqueName: \"kubernetes.io/projected/8868369b-4bd8-4d79-abcb-4ac42db6fa0f-kube-api-access-826kl\") pod \"monitoring-plugin-5f68df7f58-plj97\" (UID: \"8868369b-4bd8-4d79-abcb-4ac42db6fa0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.479779 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.479731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8868369b-4bd8-4d79-abcb-4ac42db6fa0f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-plj97\" (UID: \"8868369b-4bd8-4d79-abcb-4ac42db6fa0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.510719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.510692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-826kl\" (UniqueName: \"kubernetes.io/projected/8868369b-4bd8-4d79-abcb-4ac42db6fa0f-kube-api-access-826kl\") pod \"monitoring-plugin-5f68df7f58-plj97\" (UID: \"8868369b-4bd8-4d79-abcb-4ac42db6fa0f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.604145 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.603943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:41.618161 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.618090    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fb696657c-dww2m"]
Jul 09 18:04:41.648241 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:41.648176    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235c39b9_9046_4a1d_abbb_210284c39933.slice/crio-6c28ff2c28040535899c40d17e322aa64edcee2783c8ff7a361303be75f24fd7 WatchSource:0}: Error finding container 6c28ff2c28040535899c40d17e322aa64edcee2783c8ff7a361303be75f24fd7: Status 404 returned error can't find the container with id 6c28ff2c28040535899c40d17e322aa64edcee2783c8ff7a361303be75f24fd7
Jul 09 18:04:41.718971 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.718444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:41.838327 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.838274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"]
Jul 09 18:04:41.842377 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:41.842345    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8868369b_4bd8_4d79_abcb_4ac42db6fa0f.slice/crio-7350296a98b8b8dd5eddce619db70a732685439ff236b3bb4f6996454e770cb6 WatchSource:0}: Error finding container 7350296a98b8b8dd5eddce619db70a732685439ff236b3bb4f6996454e770cb6: Status 404 returned error can't find the container with id 7350296a98b8b8dd5eddce619db70a732685439ff236b3bb4f6996454e770cb6
Jul 09 18:04:41.935806 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:41.935764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:42.029376 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.029261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:42.141741 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.141699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54" event={"ID":"3255187c-5222-4fbb-89a7-0507b85ea06e","Type":"ContainerStarted","Data":"91d2bdd1e082c51aed9c15f457dcc622baeb723e55909f0067f374e58c036eca"}
Jul 09 18:04:42.141917 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.141777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54" event={"ID":"3255187c-5222-4fbb-89a7-0507b85ea06e","Type":"ContainerStarted","Data":"e902ad2eaa88b7a245ce19cc17c2054ed95a6036e92a31e63096fbf4012fd946"}
Jul 09 18:04:42.141917 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.141806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54" event={"ID":"3255187c-5222-4fbb-89a7-0507b85ea06e","Type":"ContainerStarted","Data":"33410343114df962c48a4545bb7e6665842d0cd8e22ea60f2047132d8b2d70e0"}
Jul 09 18:04:42.143664 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.143600    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7c7ll" event={"ID":"c1c19302-1cda-47c9-94aa-ac799575abe2","Type":"ContainerStarted","Data":"d57440c29cdbb42f0b138352968ebbbf914f847535f65b410611be4fd8455198"}
Jul 09 18:04:42.145323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.145298    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e" exitCode=0
Jul 09 18:04:42.145437 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.145375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"}
Jul 09 18:04:42.146675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.146625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97" event={"ID":"8868369b-4bd8-4d79-abcb-4ac42db6fa0f","Type":"ContainerStarted","Data":"7350296a98b8b8dd5eddce619db70a732685439ff236b3bb4f6996454e770cb6"}
Jul 09 18:04:42.147959 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.147936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m" event={"ID":"235c39b9-9046-4a1d-abbb-210284c39933","Type":"ContainerStarted","Data":"6c28ff2c28040535899c40d17e322aa64edcee2783c8ff7a361303be75f24fd7"}
Jul 09 18:04:42.149986 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.149962    2581 generic.go:356] "Generic (PLEG): container finished" podID="2e2874ca-13c1-432b-b37c-2ebe4fc1fc03" containerID="9d54529bbfdea1e215baa9ff88e07fb1c7c30b6dbe7dd653a70964195a4fb513" exitCode=0
Jul 09 18:04:42.150097 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.150046    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8tnlx" event={"ID":"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03","Type":"ContainerDied","Data":"9d54529bbfdea1e215baa9ff88e07fb1c7c30b6dbe7dd653a70964195a4fb513"}
Jul 09 18:04:42.153221 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.153176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5" event={"ID":"6fb29d9d-a69b-4ba4-92a5-dacad1c693aa","Type":"ContainerStarted","Data":"c385f4f10ca549694e659cde33cc7d68336bed7f8364b30f7306f329a253f581"}
Jul 09 18:04:42.154592 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.154574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"5bf1842e5d47949e51d3513206947de40a2fe3dc11d15d85cc5c91be8cc00074"}
Jul 09 18:04:42.211208 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.211131    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7c7ll" podStartSLOduration=66.730569553 podStartE2EDuration="1m15.211107516s" totalImagesPullingTime="8.480537963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:42.209892113 +0000 UTC m=+76.133436585" watchObservedRunningTime="2026-07-09 18:04:42.211107516 +0000 UTC m=+76.134651991"
Jul 09 18:04:42.211720 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.211681    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-cmr54" podStartSLOduration=1.8354584790000001 podStartE2EDuration="6.21166675s" totalImagesPullingTime="4.376208271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:42.176934575 +0000 UTC m=+76.100479073" watchObservedRunningTime="2026-07-09 18:04:42.21166675 +0000 UTC m=+76.135211223"
Jul 09 18:04:42.239752 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.239501    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kpdm5" podStartSLOduration=2.360015566 podStartE2EDuration="6.239479362s" totalImagesPullingTime="3.879463796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:42.238147481 +0000 UTC m=+76.161691992" watchObservedRunningTime="2026-07-09 18:04:42.239479362 +0000 UTC m=+76.163023836"
Jul 09 18:04:42.982282 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.981008    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:04:42.982282 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:42.981254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:43.004244 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.004215    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.018528 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.017340    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:04:43.018528 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.017553    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 18:04:43.018528 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.018124    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 18:04:43.018528 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.018358    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 18:04:43.025342 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.025315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4dtqgo91pchen\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.025546    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gzjcd\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.025569    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.025822    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.025831    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.026007    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.026064    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.026251    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.026296    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.026448    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 18:04:43.026884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.026616    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 18:04:43.028947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.028916    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 18:04:43.035573 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.035549    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 18:04:43.089317 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.089500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tzvv\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-kube-api-access-7tzvv\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.089500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.089500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.089500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.089500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.089500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-config\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.090628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.089908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.168615 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.168533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8tnlx" event={"ID":"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03","Type":"ContainerStarted","Data":"ff870b5b51d5328f0e63d74e0bd3f779e215b0335f3c302bdaf2ead5a25ef4e1"}
Jul 09 18:04:43.168929 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.168860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8tnlx" event={"ID":"2e2874ca-13c1-432b-b37c-2ebe4fc1fc03","Type":"ContainerStarted","Data":"8d47acb25c4a8e3bde5938fc045cd59b71bd9cc8b9e873a166813b63b8e781b7"}
Jul 09 18:04:43.191078 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191294 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191099    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191294 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191294 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tzvv\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-kube-api-access-7tzvv\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191294 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191294 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191294 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-config\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191402    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.191609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.192158 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.191623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.193004 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.192700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.193004 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.192888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.196218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.193966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.196218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.194984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.196385 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.196242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.197588 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.196764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.197588 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.196959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.199901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.200325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.200737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.200921    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-config\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.201260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.201701    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.201954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.202302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.202743    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.203614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.204561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.204372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.206435 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.206411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.241853 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.241780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tzvv\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-kube-api-access-7tzvv\") pod \"prometheus-k8s-0\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.256395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.255048    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8tnlx" podStartSLOduration=3.304800583 podStartE2EDuration="7.255025999s" totalImagesPullingTime="3.83414742s" totalInitContainerRuntime="116.077996ms" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:43.252003325 +0000 UTC m=+77.175547799" watchObservedRunningTime="2026-07-09 18:04:43.255025999 +0000 UTC m=+77.178570456"
Jul 09 18:04:43.308831 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.308792    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:43.332074 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.331753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:04:43.525585 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:43.525473    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:45.478309 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:45.478148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:04:45.479437 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:04:45.479408    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9a3a75_886f_4bdc_9772_6888f73a749f.slice/crio-615ea9359b8ed07e80f657ed8461f323ff57ab63019553f277777c6b5a33230d WatchSource:0}: Error finding container 615ea9359b8ed07e80f657ed8461f323ff57ab63019553f277777c6b5a33230d: Status 404 returned error can't find the container with id 615ea9359b8ed07e80f657ed8461f323ff57ab63019553f277777c6b5a33230d
Jul 09 18:04:45.586116 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:45.585982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:45.721419 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:45.721373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:46.185157 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.185116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"48cc22d15841ce8f1e102bd07b69b4fdf69061dbdfc0e73b10acae3b5089bc97"}
Jul 09 18:04:46.185357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.185222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"a1a6eb25b1cc218a09a75b1679fc6a9ce0b1bad8d9a2ed07b999d26ded06251a"}
Jul 09 18:04:46.185357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.185265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"5a328e896af5b7be367be661309b016da7289a4a5dc2f3c84edcd64e0222a887"}
Jul 09 18:04:46.187090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.187062    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859" exitCode=0
Jul 09 18:04:46.187210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.187147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"}
Jul 09 18:04:46.187265 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.187248    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"615ea9359b8ed07e80f657ed8461f323ff57ab63019553f277777c6b5a33230d"}
Jul 09 18:04:46.190857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.190836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"}
Jul 09 18:04:46.190952 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.190908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"}
Jul 09 18:04:46.191002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.190953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"}
Jul 09 18:04:46.191002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.190990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"}
Jul 09 18:04:46.191085 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.191030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"}
Jul 09 18:04:46.192507 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.192481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97" event={"ID":"8868369b-4bd8-4d79-abcb-4ac42db6fa0f","Type":"ContainerStarted","Data":"14246a1ac31f3ec88a120032b3cc27acca1988479704dd6d1c850c539b61cdd3"}
Jul 09 18:04:46.192675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.192655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:46.194517 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.194482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m" event={"ID":"235c39b9-9046-4a1d-abbb-210284c39933","Type":"ContainerStarted","Data":"edb1dbe50867403237bbe20d63b3175f52518a35025168c786dae5e1053c2518"}
Jul 09 18:04:46.198098 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.198077    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97"
Jul 09 18:04:46.256964 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.256895    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m" podStartSLOduration=2.644902781 podStartE2EDuration="6.256877309s" totalImagesPullingTime="3.611974528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:46.256207067 +0000 UTC m=+80.179751541" watchObservedRunningTime="2026-07-09 18:04:46.256877309 +0000 UTC m=+80.180421780"
Jul 09 18:04:46.277951 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:46.277895    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-plj97" podStartSLOduration=1.855152338 podStartE2EDuration="5.277874744s" totalImagesPullingTime="3.422722406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:46.276420871 +0000 UTC m=+80.199965343" watchObservedRunningTime="2026-07-09 18:04:46.277874744 +0000 UTC m=+80.201419218"
Jul 09 18:04:47.202497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.202384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"46f1a7d2a315a936544e33bd6a16d3701f8409e0eaea573b0e705b38428f0efd"}
Jul 09 18:04:47.202497 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.202483    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"605945fd6f7f00ef6188353af201abf02e5dbefbd3edf653e29eafbcf0045c70"}
Jul 09 18:04:47.202981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.202526    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" event={"ID":"a31a3680-a10f-43bc-a288-54b6bc6f8035","Type":"ContainerStarted","Data":"19ec61757f5cd4b03127d1315de2e8c66c5246b145b7538ed67969f9eb26995c"}
Jul 09 18:04:47.202981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.202567    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:47.206075 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.206037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerStarted","Data":"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"}
Jul 09 18:04:47.246662 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.246603    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86" podStartSLOduration=3.225762569 podStartE2EDuration="8.246581118s" totalImagesPullingTime="5.020818549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:47.246098015 +0000 UTC m=+81.169642488" watchObservedRunningTime="2026-07-09 18:04:47.246581118 +0000 UTC m=+81.170125590"
Jul 09 18:04:47.287841 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:47.287773    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.9997585730000003 podStartE2EDuration="10.28774981s" totalImagesPullingTime="7.263988463s" totalInitContainerRuntime="24.002774ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:47.28684756 +0000 UTC m=+81.210392034" watchObservedRunningTime="2026-07-09 18:04:47.28774981 +0000 UTC m=+81.211294283"
Jul 09 18:04:48.977826 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:48.977675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:49.040330 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:49.040274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:49.392056 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:49.391972    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:49.493618 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:49.493579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:50.977993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:50.977955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:51.039849 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:51.039780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:53.142719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:53.142680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:53.216697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:53.216659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:53.219217 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:53.219175    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7cb5558858-grx86"
Jul 09 18:04:53.959395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:53.959351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:54.137520 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:54.137457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:04:58.262324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.262229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"}
Jul 09 18:04:58.262791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.262372    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"}
Jul 09 18:04:58.262791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.262427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"}
Jul 09 18:04:58.262791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.262469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"}
Jul 09 18:04:58.262791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.262510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"}
Jul 09 18:04:58.262791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.262552    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerStarted","Data":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"}
Jul 09 18:04:58.264152 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.264120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-f54r7" event={"ID":"6d16fd67-63e1-4a66-8088-5f8c62bd72cb","Type":"ContainerStarted","Data":"85b8161358c627fafd42f944b61d19c42c52ab3b1cd53fae495984c3f73ecaf6"}
Jul 09 18:04:58.264308 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.264284    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:58.281251 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.281228    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-f54r7"
Jul 09 18:04:58.299633 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.299576    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.9875242669999995 podStartE2EDuration="16.299556787s" totalImagesPullingTime="11.282086829s" totalInitContainerRuntime="29.945691ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:58.297849555 +0000 UTC m=+92.221394029" watchObservedRunningTime="2026-07-09 18:04:58.299556787 +0000 UTC m=+92.223101261"
Jul 09 18:04:58.332883 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:04:58.332856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:05:00.065734 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:00.065702    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:00.140532 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:00.140502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:01.179514 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:01.179471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:05:01.179929 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:01.179539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:05:01.399806 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:01.399768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:01.500340 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:01.500261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:05.576989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:05.576943    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:05.733785 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:05.733752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:06.261380 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:06.261292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:06.334303 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:06.334262    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:10.132526 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:10.132493    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m42st"
Jul 09 18:05:10.153644 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:10.153592    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-f54r7" podStartSLOduration=14.508072952 podStartE2EDuration="35.153557128s" totalImagesPullingTime="20.645484176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:58.320864995 +0000 UTC m=+92.244409467" watchObservedRunningTime="2026-07-09 18:05:10.153557128 +0000 UTC m=+104.077101602"
Jul 09 18:05:11.301317 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:11.301282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:11.456490 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:11.456450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:11.554292 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:11.554210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:16.820874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:16.820843    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4srdt_5ee0ea18-4568-47f0-a889-8f1e623f51ac/serve-healthcheck-canary/0.log"
Jul 09 18:05:18.430608 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:18.430573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:18.530715 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:18.530687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:18.654008 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:18.653982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:21.184841 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:21.184814    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:05:21.188514 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:21.188494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fb696657c-dww2m"
Jul 09 18:05:26.447583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:26.447547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:26.527815 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:26.527790    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:38.927099 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:38.927060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:43.332268 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:43.332231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:05:43.411039 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:43.411011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:05:43.468677 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:43.468653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:05:52.065093 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:52.065055    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:05:59.735739 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:05:59.735702    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:00.037336 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.037232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:00.841963 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.841935    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:06:00.842395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.842224    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="alertmanager" containerID="cri-o://727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e" gracePeriod=120
Jul 09 18:06:00.842395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.842263    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-metric" containerID="cri-o://6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc" gracePeriod=120
Jul 09 18:06:00.842395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.842302    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-web" containerID="cri-o://08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3" gracePeriod=120
Jul 09 18:06:00.842562 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.842358    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy" containerID="cri-o://0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1" gracePeriod=120
Jul 09 18:06:00.842562 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.842389    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="prom-label-proxy" containerID="cri-o://6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6" gracePeriod=120
Jul 09 18:06:00.842562 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:00.842363    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="config-reloader" containerID="cri-o://3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08" gracePeriod=120
Jul 09 18:06:01.527947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.527914    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6" exitCode=0
Jul 09 18:06:01.527947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.527940    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1" exitCode=0
Jul 09 18:06:01.527947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.527948    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08" exitCode=0
Jul 09 18:06:01.527947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.527953    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e" exitCode=0
Jul 09 18:06:01.528218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.527979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"}
Jul 09 18:06:01.528218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.528042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"}
Jul 09 18:06:01.528218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.528071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"}
Jul 09 18:06:01.528218 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:01.528094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"}
Jul 09 18:06:02.101674 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.101652    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.182860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.182830    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-main-tls\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183026 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.182887    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-main-db\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183026 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.182911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183165 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183035    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-config-volume\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183165 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183075    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-cluster-tls-config\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183165 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183103    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-tls-assets\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183163    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183211    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-trusted-ca-bundle\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183237    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-web-config\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183241    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:06:02.183354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183268    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-metrics-client-ca\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183320    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-config-out\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183643 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183367    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-web\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183643 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183398    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvnhv\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-kube-api-access-pvnhv\") pod \"e378b45a-09aa-4a63-bba0-abff6979abe7\" (UID: \"e378b45a-09aa-4a63-bba0-abff6979abe7\") "
Jul 09 18:06:02.183643 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183606    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:02.183827 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.183649    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-main-db\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.184311 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.184032    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:02.185843 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.185797    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-config-volume" (OuterVolumeSpecName: "config-volume") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.186281 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.186241    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.187471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.187431    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-kube-api-access-pvnhv" (OuterVolumeSpecName: "kube-api-access-pvnhv") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "kube-api-access-pvnhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:06:02.187766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.187699    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.187766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.187702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-config-out" (OuterVolumeSpecName: "config-out") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:06:02.187938 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.187765    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:06:02.187938 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.187891    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.188047 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.187959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.191640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.191619    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.198156 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.198136    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-web-config" (OuterVolumeSpecName: "web-config") pod "e378b45a-09aa-4a63-bba0-abff6979abe7" (UID: "e378b45a-09aa-4a63-bba0-abff6979abe7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:02.284650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284610    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-main-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284646    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284662    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-config-volume\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284677    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-cluster-tls-config\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284687    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-tls-assets\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284700    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284713    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284751    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-web-config\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284764    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e378b45a-09aa-4a63-bba0-abff6979abe7-metrics-client-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284777    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e378b45a-09aa-4a63-bba0-abff6979abe7-config-out\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284792    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e378b45a-09aa-4a63-bba0-abff6979abe7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.284814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.284807    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvnhv\" (UniqueName: \"kubernetes.io/projected/e378b45a-09aa-4a63-bba0-abff6979abe7-kube-api-access-pvnhv\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:02.534719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534641    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc" exitCode=0
Jul 09 18:06:02.534719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534667    2581 generic.go:356] "Generic (PLEG): container finished" podID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerID="08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3" exitCode=0
Jul 09 18:06:02.534719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.534923 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534720    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"}
Jul 09 18:06:02.534923 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"}
Jul 09 18:06:02.534923 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e378b45a-09aa-4a63-bba0-abff6979abe7","Type":"ContainerDied","Data":"fd6da4778c75af47d61732f939f2130bd6261f08c31461fa2083904d6ff5f26d"}
Jul 09 18:06:02.534923 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.534846    2581 scope.go:117] "RemoveContainer" containerID="6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"
Jul 09 18:06:02.543349 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.543332    2581 scope.go:117] "RemoveContainer" containerID="6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"
Jul 09 18:06:02.550353 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.550336    2581 scope.go:117] "RemoveContainer" containerID="0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"
Jul 09 18:06:02.556951 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.556933    2581 scope.go:117] "RemoveContainer" containerID="08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"
Jul 09 18:06:02.558868 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.558844    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:06:02.564509 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.564491    2581 scope.go:117] "RemoveContainer" containerID="3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"
Jul 09 18:06:02.565673 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.565654    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:06:02.571030 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.571016    2581 scope.go:117] "RemoveContainer" containerID="727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"
Jul 09 18:06:02.577719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.577702    2581 scope.go:117] "RemoveContainer" containerID="699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"
Jul 09 18:06:02.584483 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.584465    2581 scope.go:117] "RemoveContainer" containerID="6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"
Jul 09 18:06:02.584755 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.584733    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6\": container with ID starting with 6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6 not found: ID does not exist" containerID="6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"
Jul 09 18:06:02.584798 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.584764    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"} err="failed to get container status \"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6\": rpc error: code = NotFound desc = could not find container \"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6\": container with ID starting with 6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6 not found: ID does not exist"
Jul 09 18:06:02.584798 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.584793    2581 scope.go:117] "RemoveContainer" containerID="6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"
Jul 09 18:06:02.585027 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.585009    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc\": container with ID starting with 6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc not found: ID does not exist" containerID="6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"
Jul 09 18:06:02.585066 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585034    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"} err="failed to get container status \"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc\": rpc error: code = NotFound desc = could not find container \"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc\": container with ID starting with 6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc not found: ID does not exist"
Jul 09 18:06:02.585066 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585051    2581 scope.go:117] "RemoveContainer" containerID="0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"
Jul 09 18:06:02.585289 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.585273    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1\": container with ID starting with 0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1 not found: ID does not exist" containerID="0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"
Jul 09 18:06:02.585356 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585297    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"} err="failed to get container status \"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1\": rpc error: code = NotFound desc = could not find container \"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1\": container with ID starting with 0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1 not found: ID does not exist"
Jul 09 18:06:02.585356 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585320    2581 scope.go:117] "RemoveContainer" containerID="08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"
Jul 09 18:06:02.585545 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.585529    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3\": container with ID starting with 08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3 not found: ID does not exist" containerID="08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"
Jul 09 18:06:02.585582 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585550    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"} err="failed to get container status \"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3\": rpc error: code = NotFound desc = could not find container \"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3\": container with ID starting with 08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3 not found: ID does not exist"
Jul 09 18:06:02.585582 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585564    2581 scope.go:117] "RemoveContainer" containerID="3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"
Jul 09 18:06:02.585778 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.585762    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08\": container with ID starting with 3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08 not found: ID does not exist" containerID="3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"
Jul 09 18:06:02.585819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585783    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"} err="failed to get container status \"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08\": rpc error: code = NotFound desc = could not find container \"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08\": container with ID starting with 3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08 not found: ID does not exist"
Jul 09 18:06:02.585819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.585799    2581 scope.go:117] "RemoveContainer" containerID="727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"
Jul 09 18:06:02.586033 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.586011    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e\": container with ID starting with 727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e not found: ID does not exist" containerID="727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"
Jul 09 18:06:02.586156 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586035    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"} err="failed to get container status \"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e\": rpc error: code = NotFound desc = could not find container \"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e\": container with ID starting with 727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e not found: ID does not exist"
Jul 09 18:06:02.586156 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586050    2581 scope.go:117] "RemoveContainer" containerID="699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"
Jul 09 18:06:02.586363 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:02.586267    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e\": container with ID starting with 699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e not found: ID does not exist" containerID="699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"
Jul 09 18:06:02.586363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586285    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"} err="failed to get container status \"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e\": rpc error: code = NotFound desc = could not find container \"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e\": container with ID starting with 699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e not found: ID does not exist"
Jul 09 18:06:02.586363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586306    2581 scope.go:117] "RemoveContainer" containerID="6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"
Jul 09 18:06:02.586600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586573    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6"} err="failed to get container status \"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6\": rpc error: code = NotFound desc = could not find container \"6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6\": container with ID starting with 6b8d14f36471c166f081dd49e451eeaa7aa99f010c2483cb3aa6c9a94ef8f6c6 not found: ID does not exist"
Jul 09 18:06:02.586641 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586601    2581 scope.go:117] "RemoveContainer" containerID="6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"
Jul 09 18:06:02.586835 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586818    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc"} err="failed to get container status \"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc\": rpc error: code = NotFound desc = could not find container \"6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc\": container with ID starting with 6eebc1e6fd3a0087fdb3f0d0258a97b8b1bb39ff40c90cf5e6ff24f395b5f6cc not found: ID does not exist"
Jul 09 18:06:02.586878 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.586835    2581 scope.go:117] "RemoveContainer" containerID="0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"
Jul 09 18:06:02.587039 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587021    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1"} err="failed to get container status \"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1\": rpc error: code = NotFound desc = could not find container \"0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1\": container with ID starting with 0b08c8c684b04b8c95b32e4e8ca4028255a42da097fee6c510f95dfddee299a1 not found: ID does not exist"
Jul 09 18:06:02.587105 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587041    2581 scope.go:117] "RemoveContainer" containerID="08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"
Jul 09 18:06:02.587266 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587248    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3"} err="failed to get container status \"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3\": rpc error: code = NotFound desc = could not find container \"08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3\": container with ID starting with 08b52b731066ff20cc635139cff2f1b4c90a74fc54720856abc7ebdab3fdd9c3 not found: ID does not exist"
Jul 09 18:06:02.587339 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587268    2581 scope.go:117] "RemoveContainer" containerID="3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"
Jul 09 18:06:02.587463 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587446    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08"} err="failed to get container status \"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08\": rpc error: code = NotFound desc = could not find container \"3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08\": container with ID starting with 3091cab1bd88e4c39ff8c3900a827bd5f97bbc8b47f7d6873b39c3982c9b6f08 not found: ID does not exist"
Jul 09 18:06:02.587527 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587465    2581 scope.go:117] "RemoveContainer" containerID="727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"
Jul 09 18:06:02.587680 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587662    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e"} err="failed to get container status \"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e\": rpc error: code = NotFound desc = could not find container \"727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e\": container with ID starting with 727cda10a93647bdffe81165cee6e0b6e6958aeef3831958e2e69cb860766a2e not found: ID does not exist"
Jul 09 18:06:02.587747 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587682    2581 scope.go:117] "RemoveContainer" containerID="699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"
Jul 09 18:06:02.587896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.587878    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e"} err="failed to get container status \"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e\": rpc error: code = NotFound desc = could not find container \"699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e\": container with ID starting with 699f6d9bd37641f5a49452d6978dae915170f1772415de0a5694b0de5b08c06e not found: ID does not exist"
Jul 09 18:06:02.591658 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591636    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:06:02.591924 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591913    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="prom-label-proxy"
Jul 09 18:06:02.591964 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591925    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="prom-label-proxy"
Jul 09 18:06:02.591964 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591939    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="alertmanager"
Jul 09 18:06:02.591964 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591944    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="alertmanager"
Jul 09 18:06:02.591964 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591957    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-web"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591965    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-web"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591977    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-metric"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591986    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-metric"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591994    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.591999    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592005    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="init-config-reloader"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592010    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="init-config-reloader"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592019    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="config-reloader"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592024    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="config-reloader"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592070    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy"
Jul 09 18:06:02.592077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592078    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-metric"
Jul 09 18:06:02.592396 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592086    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="config-reloader"
Jul 09 18:06:02.592396 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592092    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="alertmanager"
Jul 09 18:06:02.592396 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592099    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="prom-label-proxy"
Jul 09 18:06:02.592396 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.592106    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" containerName="kube-rbac-proxy-web"
Jul 09 18:06:02.595610 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.595593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.597872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.597856    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 18:06:02.598124 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598108    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 18:06:02.598395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 18:06:02.598395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 18:06:02.598537 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598405    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 18:06:02.598537 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598504    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 18:06:02.598537 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598508    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 18:06:02.598537 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598528    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 18:06:02.598831 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.598818    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dlv55\""
Jul 09 18:06:02.605120 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.604626    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 18:06:02.617204 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.617163    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:06:02.687999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.687968    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.687999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fbe37bc-a838-4e1b-8749-935e68bef66a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688220 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fbe37bc-a838-4e1b-8749-935e68bef66a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688220 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688220 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688220 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7fbe37bc-a838-4e1b-8749-935e68bef66a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688220 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r65w\" (UniqueName: \"kubernetes.io/projected/7fbe37bc-a838-4e1b-8749-935e68bef66a-kube-api-access-8r65w\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-config-volume\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7fbe37bc-a838-4e1b-8749-935e68bef66a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-web-config\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.688405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.688382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7fbe37bc-a838-4e1b-8749-935e68bef66a-config-out\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789380 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789380 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fbe37bc-a838-4e1b-8749-935e68bef66a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789380 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fbe37bc-a838-4e1b-8749-935e68bef66a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789380 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7fbe37bc-a838-4e1b-8749-935e68bef66a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r65w\" (UniqueName: \"kubernetes.io/projected/7fbe37bc-a838-4e1b-8749-935e68bef66a-kube-api-access-8r65w\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-config-volume\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7fbe37bc-a838-4e1b-8749-935e68bef66a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-web-config\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.789665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.789629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7fbe37bc-a838-4e1b-8749-935e68bef66a-config-out\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.790144 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.790116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fbe37bc-a838-4e1b-8749-935e68bef66a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.790352 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.790286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7fbe37bc-a838-4e1b-8749-935e68bef66a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.790576 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.790550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fbe37bc-a838-4e1b-8749-935e68bef66a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.792544 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.792516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7fbe37bc-a838-4e1b-8749-935e68bef66a-config-out\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.792700 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.792678    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.793169 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.792755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.793169 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.792832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7fbe37bc-a838-4e1b-8749-935e68bef66a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.793169 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.793049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.793169 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.793130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-web-config\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.793169 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.793162    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.793393 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.793327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-config-volume\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.794008 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.793988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7fbe37bc-a838-4e1b-8749-935e68bef66a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.798282 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.798259    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e378b45a-09aa-4a63-bba0-abff6979abe7" path="/var/lib/kubelet/pods/e378b45a-09aa-4a63-bba0-abff6979abe7/volumes"
Jul 09 18:06:02.799393 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.799374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r65w\" (UniqueName: \"kubernetes.io/projected/7fbe37bc-a838-4e1b-8749-935e68bef66a-kube-api-access-8r65w\") pod \"alertmanager-main-0\" (UID: \"7fbe37bc-a838-4e1b-8749-935e68bef66a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:02.910499 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:02.910471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 18:06:03.055132 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:03.054983    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 18:06:03.057625 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:06:03.057600    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fbe37bc_a838_4e1b_8749_935e68bef66a.slice/crio-c471f52dc7c91beb8337ec47eeeec135d3b746ed2ca891b9d8885f35ad398224 WatchSource:0}: Error finding container c471f52dc7c91beb8337ec47eeeec135d3b746ed2ca891b9d8885f35ad398224: Status 404 returned error can't find the container with id c471f52dc7c91beb8337ec47eeeec135d3b746ed2ca891b9d8885f35ad398224
Jul 09 18:06:03.540083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:03.540052    2581 generic.go:356] "Generic (PLEG): container finished" podID="7fbe37bc-a838-4e1b-8749-935e68bef66a" containerID="67e2859922b09895250dcbd6a0407eeaa51345a65916c5b0fe64009e3c00fe11" exitCode=0
Jul 09 18:06:03.540491 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:03.540139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerDied","Data":"67e2859922b09895250dcbd6a0407eeaa51345a65916c5b0fe64009e3c00fe11"}
Jul 09 18:06:03.540491 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:03.540241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"c471f52dc7c91beb8337ec47eeeec135d3b746ed2ca891b9d8885f35ad398224"}
Jul 09 18:06:04.548919 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.548877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"64349e1287ecf521aa27b4ded655929bc390af8752def683ff9caa997cc1e59f"}
Jul 09 18:06:04.549274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.548957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"739988c86800265379f015c7f28ad99ef562f9c2e1eb0c0d9137c6628ac4e894"}
Jul 09 18:06:04.549274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.549000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"6eaf0248877ee11cd227fdbcbef1bf341584e74f455739aabd68d9de0d64fe9d"}
Jul 09 18:06:04.549274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.549041    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"946c53923c81a54cded25300b41f9e2974cd8e580977cd8b218c5b640af2cb0c"}
Jul 09 18:06:04.549274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.549082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"f0d407e3b40b92fed5e7b399a1dd3a14fa4001862ddb2ce3e42e1a3c6619fb55"}
Jul 09 18:06:04.549274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.549116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7fbe37bc-a838-4e1b-8749-935e68bef66a","Type":"ContainerStarted","Data":"262023fc3fc60056c17e5586158d7384d0b85ae6222d83fa4c26905e8d629dae"}
Jul 09 18:06:04.577195 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.577143    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.563359464 podStartE2EDuration="2.577128297s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.768833ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:06:04.575911767 +0000 UTC m=+158.499456239" watchObservedRunningTime="2026-07-09 18:06:04.577128297 +0000 UTC m=+158.500672768"
Jul 09 18:06:04.869856 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.869819    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7695ff9586-d576f"]
Jul 09 18:06:04.872329 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.872298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.874880 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.874850    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 18:06:04.875357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.875332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qshpg\""
Jul 09 18:06:04.875594 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.875571    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 18:06:04.875685 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.875673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 18:06:04.875845 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.875826    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 18:06:04.876119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.876098    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 18:06:04.881299 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.881276    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 18:06:04.889728 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.889708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7695ff9586-d576f"]
Jul 09 18:06:04.907567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-secret-telemeter-client\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfq9h\" (UniqueName: \"kubernetes.io/projected/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-kube-api-access-vfq9h\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-telemeter-client-tls\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-federate-client-tls\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-metrics-client-ca\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:04.907963 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:04.907834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-serving-certs-ca-bundle\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009061 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009259 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009259 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-federate-client-tls\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009259 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-metrics-client-ca\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009259 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-serving-certs-ca-bundle\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009465 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-secret-telemeter-client\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009465 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfq9h\" (UniqueName: \"kubernetes.io/projected/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-kube-api-access-vfq9h\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009465 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-telemeter-client-tls\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.009958 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.009927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-metrics-client-ca\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.010062 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.010005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.010062 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.010021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-serving-certs-ca-bundle\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.012148 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.012125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-telemeter-client-tls\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.012247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.012159    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.012327 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.012309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-secret-telemeter-client\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.012610 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.012593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-federate-client-tls\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.017811 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.017789    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfq9h\" (UniqueName: \"kubernetes.io/projected/3f0d55cd-cf73-445b-a342-41cf3c46fe3d-kube-api-access-vfq9h\") pod \"telemeter-client-7695ff9586-d576f\" (UID: \"3f0d55cd-cf73-445b-a342-41cf3c46fe3d\") " pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.152333 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152238    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:06:05.152597 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152558    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="prometheus" containerID="cri-o://f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf" gracePeriod=600
Jul 09 18:06:05.152823 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152579    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy" containerID="cri-o://874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527" gracePeriod=600
Jul 09 18:06:05.152823 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152602    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-web" containerID="cri-o://3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790" gracePeriod=600
Jul 09 18:06:05.152823 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152624    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="config-reloader" containerID="cri-o://17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf" gracePeriod=600
Jul 09 18:06:05.152823 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152620    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="thanos-sidecar" containerID="cri-o://5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467" gracePeriod=600
Jul 09 18:06:05.152823 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.152786    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6" gracePeriod=600
Jul 09 18:06:05.184900 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.184830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f"
Jul 09 18:06:05.338037 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.337892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7695ff9586-d576f"]
Jul 09 18:06:05.345064 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:06:05.345037    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0d55cd_cf73_445b_a342_41cf3c46fe3d.slice/crio-1f16f232858a0fc7236ead226020313f0e9a6f38575e08321d2b2e723fc72843 WatchSource:0}: Error finding container 1f16f232858a0fc7236ead226020313f0e9a6f38575e08321d2b2e723fc72843: Status 404 returned error can't find the container with id 1f16f232858a0fc7236ead226020313f0e9a6f38575e08321d2b2e723fc72843
Jul 09 18:06:05.407122 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.407054    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.514513 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514483    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-trusted-ca-bundle\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514698 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514521    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-metrics-client-certs\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514698 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-config\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514698 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514586    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-config-out\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514698 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514631    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-kube-rbac-proxy\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514698 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514661    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-metrics-client-ca\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514746    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tzvv\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-kube-api-access-7tzvv\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514787    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-serving-certs-ca-bundle\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514896    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-web-config\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514930    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-grpc-tls\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514932    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.514965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514957    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.514990    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-1\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515016    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-tls-assets\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515058    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-tls\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515086    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-0\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515117    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-thanos-prometheus-http-client-file\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515116    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-2\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515210    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-kubelet-serving-ca-bundle\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515242    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-db\") pod \"5c9a3a75-886f-4bdc-9772-6888f73a749f\" (UID: \"5c9a3a75-886f-4bdc-9772-6888f73a749f\") "
Jul 09 18:06:05.515796 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515562    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-metrics-client-ca\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.515796 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.515584    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.516956 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.516924    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:06:05.517439 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517178    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-config" (OuterVolumeSpecName: "config") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.517439 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517228    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.517621 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517482    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-config-out" (OuterVolumeSpecName: "config-out") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:06:05.517621 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517551    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.517621 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517566    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.517936 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517899    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.518010 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.517960    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.518372 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.518347    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.518470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.518412    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.518756 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.518733    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.519061 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.519031    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 18:06:05.519263 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.519244    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.519484 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.519466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-kube-api-access-7tzvv" (OuterVolumeSpecName: "kube-api-access-7tzvv") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "kube-api-access-7tzvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:06:05.519995 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.519972    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:06:05.520130 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.520117    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.520617 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.520595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.529472 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.529452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-web-config" (OuterVolumeSpecName: "web-config") pod "5c9a3a75-886f-4bdc-9772-6888f73a749f" (UID: "5c9a3a75-886f-4bdc-9772-6888f73a749f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:06:05.556383 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556361    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6" exitCode=0
Jul 09 18:06:05.556383 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556381    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527" exitCode=0
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556387    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790" exitCode=0
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556393    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467" exitCode=0
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556399    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf" exitCode=0
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556404    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf" exitCode=0
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556445    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"}
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"}
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556564    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"}
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"}
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556656    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"}
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"}
Jul 09 18:06:05.556753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.556739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c9a3a75-886f-4bdc-9772-6888f73a749f","Type":"ContainerDied","Data":"615ea9359b8ed07e80f657ed8461f323ff57ab63019553f277777c6b5a33230d"}
Jul 09 18:06:05.557990 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.557964    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f" event={"ID":"3f0d55cd-cf73-445b-a342-41cf3c46fe3d","Type":"ContainerStarted","Data":"1f16f232858a0fc7236ead226020313f0e9a6f38575e08321d2b2e723fc72843"}
Jul 09 18:06:05.564997 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.564980    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.572239 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.572222    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.578631 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.578612    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.583899 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.583881    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:06:05.586030 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.586013    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.589549 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.589529    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:06:05.593090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.593076    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.600067 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.600049    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.608657 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.608640    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.608922 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.608901    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.608999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.608929    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"} err="failed to get container status \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist"
Jul 09 18:06:05.608999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.608946    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.609163 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.609150    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.609221 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609167    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"} err="failed to get container status \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist"
Jul 09 18:06:05.609221 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609181    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.609412 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.609397    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.609449 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609415    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"} err="failed to get container status \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist"
Jul 09 18:06:05.609449 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609428    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.609655 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.609637    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.609697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609661    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"} err="failed to get container status \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist"
Jul 09 18:06:05.609697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609676    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.609878 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.609861    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.609924 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609883    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"} err="failed to get container status \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist"
Jul 09 18:06:05.609924 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.609899    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.610130 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.610114    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.610176 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610133    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"} err="failed to get container status \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist"
Jul 09 18:06:05.610176 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610147    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.610403 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:06:05.610381    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.610496 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610413    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"} err="failed to get container status \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist"
Jul 09 18:06:05.610496 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610432    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.610654 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610636    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"} err="failed to get container status \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist"
Jul 09 18:06:05.610696 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610654    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.610889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610864    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"} err="failed to get container status \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist"
Jul 09 18:06:05.610889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.610885    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.611115 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611090    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"} err="failed to get container status \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist"
Jul 09 18:06:05.611166 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611121    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.611469 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611441    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"} err="failed to get container status \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist"
Jul 09 18:06:05.611554 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611466    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.611736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611715    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"} err="failed to get container status \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist"
Jul 09 18:06:05.611787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611737    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.611998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611980    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"} err="failed to get container status \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist"
Jul 09 18:06:05.612063 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.611999    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.612240 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612223    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"} err="failed to get container status \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist"
Jul 09 18:06:05.612300 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612240    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.612456 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612439    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"} err="failed to get container status \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist"
Jul 09 18:06:05.612456 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612455    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.612673 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612658    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"} err="failed to get container status \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist"
Jul 09 18:06:05.612673 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612672    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.612885 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612868    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"} err="failed to get container status \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist"
Jul 09 18:06:05.612934 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.612886    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.613123 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613105    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"} err="failed to get container status \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist"
Jul 09 18:06:05.613166 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613126    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.613352 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613334    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"} err="failed to get container status \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist"
Jul 09 18:06:05.613390 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613352    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.613541 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613524    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"} err="failed to get container status \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist"
Jul 09 18:06:05.613585 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613542    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.613720 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613699    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"} err="failed to get container status \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist"
Jul 09 18:06:05.613760 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613721    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.613890 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613875    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"} err="failed to get container status \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist"
Jul 09 18:06:05.613931 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.613890    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.614058 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614043    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"} err="failed to get container status \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist"
Jul 09 18:06:05.614099 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614058    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.614274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614254    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"} err="failed to get container status \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist"
Jul 09 18:06:05.614356 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614276    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.614481 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614464    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"} err="failed to get container status \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist"
Jul 09 18:06:05.614527 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614481    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.614688 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614670    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"} err="failed to get container status \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist"
Jul 09 18:06:05.614758 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614689    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.614896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614880    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"} err="failed to get container status \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist"
Jul 09 18:06:05.614942 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.614897    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.615060 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615045    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"} err="failed to get container status \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist"
Jul 09 18:06:05.615114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615060    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.615267 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615250    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"} err="failed to get container status \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist"
Jul 09 18:06:05.615267 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615267    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.615459 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615437    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"} err="failed to get container status \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist"
Jul 09 18:06:05.615500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615461    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.615649 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615633    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"} err="failed to get container status \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist"
Jul 09 18:06:05.615686 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615650    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.615875 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615858    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"} err="failed to get container status \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist"
Jul 09 18:06:05.615928 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.615875    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616104    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616129    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-web-config\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616143    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-grpc-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616157    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616171    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616242    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-tls-assets\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616270    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616284    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616299    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616315    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616329    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616135    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"} err="failed to get container status \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist"
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616345    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616352    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-prometheus-k8s-db\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616368    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-metrics-client-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616382    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-config\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616395    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c9a3a75-886f-4bdc-9772-6888f73a749f-config-out\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616410    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c9a3a75-886f-4bdc-9772-6888f73a749f-secret-kube-rbac-proxy\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616422    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tzvv\" (UniqueName: \"kubernetes.io/projected/5c9a3a75-886f-4bdc-9772-6888f73a749f-kube-api-access-7tzvv\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616434    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c9a3a75-886f-4bdc-9772-6888f73a749f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616574    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"} err="failed to get container status \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist"
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616592    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616794    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"} err="failed to get container status \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist"
Jul 09 18:06:05.616993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.616819    2581 scope.go:117] "RemoveContainer" containerID="3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"
Jul 09 18:06:05.617344 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617068    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6"} err="failed to get container status \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": rpc error: code = NotFound desc = could not find container \"3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6\": container with ID starting with 3d7025eacd70e9d28598051fefe1d749153e2a8c1f60ce3849bcc8fdb5c17de6 not found: ID does not exist"
Jul 09 18:06:05.617344 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617085    2581 scope.go:117] "RemoveContainer" containerID="874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"
Jul 09 18:06:05.617344 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617303    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527"} err="failed to get container status \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": rpc error: code = NotFound desc = could not find container \"874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527\": container with ID starting with 874d19423c81c1bacdf191d8cba1501e080a4f6cd777833272e91d53cc418527 not found: ID does not exist"
Jul 09 18:06:05.617344 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617316    2581 scope.go:117] "RemoveContainer" containerID="3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"
Jul 09 18:06:05.617520 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617503    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790"} err="failed to get container status \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": rpc error: code = NotFound desc = could not find container \"3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790\": container with ID starting with 3ec18d2181a808c65a2248561207282bd433e034c91d732e885528e63c930790 not found: ID does not exist"
Jul 09 18:06:05.617560 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617521    2581 scope.go:117] "RemoveContainer" containerID="5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"
Jul 09 18:06:05.617705 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617690    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467"} err="failed to get container status \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": rpc error: code = NotFound desc = could not find container \"5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467\": container with ID starting with 5b68ddb138d9238089b83b1d22afa8327ae9d930a2eeaa6235fc823e1d228467 not found: ID does not exist"
Jul 09 18:06:05.617753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617704    2581 scope.go:117] "RemoveContainer" containerID="17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"
Jul 09 18:06:05.617872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617855    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf"} err="failed to get container status \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": rpc error: code = NotFound desc = could not find container \"17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf\": container with ID starting with 17d831c5a34601be071d5f5527e6e8d325e64735b42173aaa22c4c705a4f3dcf not found: ID does not exist"
Jul 09 18:06:05.617908 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.617872    2581 scope.go:117] "RemoveContainer" containerID="f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"
Jul 09 18:06:05.618049 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.618035    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf"} err="failed to get container status \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": rpc error: code = NotFound desc = could not find container \"f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf\": container with ID starting with f40de6d02b924be447578531591138d44eb66830cf60826e9c7aec0b65ebd8bf not found: ID does not exist"
Jul 09 18:06:05.618094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.618050    2581 scope.go:117] "RemoveContainer" containerID="5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"
Jul 09 18:06:05.618316 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.618288    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859"} err="failed to get container status \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": rpc error: code = NotFound desc = could not find container \"5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859\": container with ID starting with 5f67a9d145480c5f1df907af5a6791f0c3c9c99e1d36254864dcdc5281b62859 not found: ID does not exist"
Jul 09 18:06:05.623616 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623596    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:06:05.623906 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623892    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="config-reloader"
Jul 09 18:06:05.623945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623908    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="config-reloader"
Jul 09 18:06:05.623945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623917    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="prometheus"
Jul 09 18:06:05.623945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623923    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="prometheus"
Jul 09 18:06:05.623945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623928    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="thanos-sidecar"
Jul 09 18:06:05.623945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623933    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="thanos-sidecar"
Jul 09 18:06:05.623945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623941    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-thanos"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623947    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-thanos"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623967    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-web"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623972    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-web"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623978    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="init-config-reloader"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623983    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="init-config-reloader"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623988    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.623993    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.624045    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.624056    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="prometheus"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.624061    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="config-reloader"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.624066    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-thanos"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.624071    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="thanos-sidecar"
Jul 09 18:06:05.624111 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.624077    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" containerName="kube-rbac-proxy-web"
Jul 09 18:06:05.628066 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.628051    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.633615 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.633597    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 18:06:05.634649 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.634629    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4dtqgo91pchen\""
Jul 09 18:06:05.634927 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.634911    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gzjcd\""
Jul 09 18:06:05.635114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.635092    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 18:06:05.635359 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.635278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.635795    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.635851    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.636011    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.636086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.636495    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.636563    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.636711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 18:06:05.637053 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.636929    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 18:06:05.637571 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.637416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 18:06:05.640581 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.640466    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 18:06:05.646171 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.646131    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 18:06:05.648665 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.648643    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:06:05.717313 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717234    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82eb4f29-899a-427b-aff2-98ee9062506f-config-out\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717313 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-web-config\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717313 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82eb4f29-899a-427b-aff2-98ee9062506f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz8ll\" (UniqueName: \"kubernetes.io/projected/82eb4f29-899a-427b-aff2-98ee9062506f-kube-api-access-lz8ll\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717538    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-config\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717653    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717699 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717952 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.717952 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.717734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818535 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818508    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-config\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82eb4f29-899a-427b-aff2-98ee9062506f-config-out\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-web-config\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.818887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82eb4f29-899a-427b-aff2-98ee9062506f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.818986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.819008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz8ll\" (UniqueName: \"kubernetes.io/projected/82eb4f29-899a-427b-aff2-98ee9062506f-kube-api-access-lz8ll\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.819033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819290 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.819241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819893 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.819867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.819973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.819955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.820593 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.820561    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.820786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.820761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.821114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822128 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821625    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822128 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821658    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822128 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822128 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822128 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-config\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822128 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.821935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.822958 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.822931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.823578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.823553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.823720 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.823704    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-web-config\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.824132 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.824109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82eb4f29-899a-427b-aff2-98ee9062506f-config-out\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.824132 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.824120    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82eb4f29-899a-427b-aff2-98ee9062506f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.824422 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.824402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82eb4f29-899a-427b-aff2-98ee9062506f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.824726 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.824710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82eb4f29-899a-427b-aff2-98ee9062506f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.831047 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.831025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz8ll\" (UniqueName: \"kubernetes.io/projected/82eb4f29-899a-427b-aff2-98ee9062506f-kube-api-access-lz8ll\") pod \"prometheus-k8s-0\" (UID: \"82eb4f29-899a-427b-aff2-98ee9062506f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:05.938510 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:05.938483    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:06.088386 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:06.088361    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 18:06:06.089684 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:06:06.089661    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82eb4f29_899a_427b_aff2_98ee9062506f.slice/crio-c875eaf6076a3114aa432de3e48231541e778ee085f09ec49e62581f9f47c238 WatchSource:0}: Error finding container c875eaf6076a3114aa432de3e48231541e778ee085f09ec49e62581f9f47c238: Status 404 returned error can't find the container with id c875eaf6076a3114aa432de3e48231541e778ee085f09ec49e62581f9f47c238
Jul 09 18:06:06.563319 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:06.563284    2581 generic.go:356] "Generic (PLEG): container finished" podID="82eb4f29-899a-427b-aff2-98ee9062506f" containerID="a892bd39496139d59b96dc5ca767fffcd8aa8b2f65a12bad7c647a314f5729dc" exitCode=0
Jul 09 18:06:06.563724 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:06.563382    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerDied","Data":"a892bd39496139d59b96dc5ca767fffcd8aa8b2f65a12bad7c647a314f5729dc"}
Jul 09 18:06:06.563724 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:06.563469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"c875eaf6076a3114aa432de3e48231541e778ee085f09ec49e62581f9f47c238"}
Jul 09 18:06:06.803591 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:06.803444    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c9a3a75-886f-4bdc-9772-6888f73a749f" path="/var/lib/kubelet/pods/5c9a3a75-886f-4bdc-9772-6888f73a749f/volumes"
Jul 09 18:06:07.574763 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.574732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"4a8e4c909572be1972f1085739118e84c420e2e94fc4ba7b737230e42e4d1e0c"}
Jul 09 18:06:07.575210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.574848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"ccad97f29b18fc7cda93b01d04a020d6673ff5c59e139354fbac558ab77fbda8"}
Jul 09 18:06:07.575210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.574894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"24c96c8594b2a8b03a3176c89b58b0a594ae6ebac464c115c536848aac42e42b"}
Jul 09 18:06:07.575210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.574931    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"3988eb0350afc7b37ab038089c86995a8e63d8f1125f106d4958f97c86c9d0fa"}
Jul 09 18:06:07.575210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.574976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"8710d6940e16599f3502f23dc6e4d6643ed2d895fd80410ad080f92275dbd5aa"}
Jul 09 18:06:07.575210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.575020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82eb4f29-899a-427b-aff2-98ee9062506f","Type":"ContainerStarted","Data":"cb5fb5df6ee69fffe922eaa0f5a566b813749fe90e5be0a49401f80d06682bdf"}
Jul 09 18:06:07.615787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:07.615619    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.592624366 podStartE2EDuration="2.615599717s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.975351ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:06:07.613607007 +0000 UTC m=+161.537151479" watchObservedRunningTime="2026-07-09 18:06:07.615599717 +0000 UTC m=+161.539144189"
Jul 09 18:06:08.581548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:08.581510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f" event={"ID":"3f0d55cd-cf73-445b-a342-41cf3c46fe3d","Type":"ContainerStarted","Data":"0c8f15d04a1b50fc65537dcf321c4429288e22e9658ef087597db3558cb0a024"}
Jul 09 18:06:08.581998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:08.581565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f" event={"ID":"3f0d55cd-cf73-445b-a342-41cf3c46fe3d","Type":"ContainerStarted","Data":"1108024c2ff00fd01ba72f1e08bd61a58a2583a8c64459c7a76838552f69a850"}
Jul 09 18:06:08.581998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:08.581584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f" event={"ID":"3f0d55cd-cf73-445b-a342-41cf3c46fe3d","Type":"ContainerStarted","Data":"9ca86b7c5c86a80e4c919bb822190b42f292d7d5040f86e7a86bab61cce0773a"}
Jul 09 18:06:08.606268 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:08.606182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7695ff9586-d576f" podStartSLOduration=2.339951441 podStartE2EDuration="4.606165961s" totalImagesPullingTime="2.26621452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:06:08.604679993 +0000 UTC m=+162.528224464" watchObservedRunningTime="2026-07-09 18:06:08.606165961 +0000 UTC m=+162.529710424"
Jul 09 18:06:09.323476 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:09.323401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:09.403037 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:09.402969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:10.938619 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:10.938587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:06:20.524961 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:20.524870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:20.625932 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:20.625893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:20.726992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:20.726956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:46.969988 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:46.969952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:06:47.051415 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:06:47.051382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:05.939357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:05.939319    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:07:05.954147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:05.954124    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:07:06.829030 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:06.829003    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 18:07:15.070081 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:15.070044    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:15.165786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:15.165756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:23.589937 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:23.589895    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:23.652666 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:23.652634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:26.908725 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:26.908688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:26.967286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:26.967255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:34.813611 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:34.813570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:34.914870 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:34.914836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:34.972092 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:34.972063    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:41.435254 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:41.435214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:07:41.535600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:07:41.535567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:20.888997 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:20.888960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:20.986355 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:20.986326    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:21.212469 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:21.212380    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:21.315195 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:21.315148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:21.413091 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:21.413057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:26.492122 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:26.492098    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:08:26.495202 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:26.495170    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 18:08:26.504939 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:26.504923    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:08:54.545282 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:54.545250    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:08:56.146531 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:08:56.146487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:00.371685 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:00.371649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:00.875896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:00.875858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:01.113073 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:01.113041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:01.611224 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:01.611173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:01.698716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:01.698682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:01.856657 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:01.856625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:01.975695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:01.975614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:02.146979 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:02.146942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:02.408785 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:02.408751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:02.762860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:02.762779    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:02.874692 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:02.874659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:09:54.922914 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:09:54.922880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:12.885966 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:12.885932    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:13.084644 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.084606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:13.350332 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.350309    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-lrtg7"]
Jul 09 18:10:13.353607 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.353591    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.363730 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.363709    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 18:10:13.364349 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.364332    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 18:10:13.364446 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.364349    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 18:10:13.364446 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.364352    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-qftwd\""
Jul 09 18:10:13.377809 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.377784    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-lrtg7"]
Jul 09 18:10:13.424600 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.424566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3ecd8a36-969e-4f19-837b-e4c164de1b50-data\") pod \"seaweedfs-64568bcd49-lrtg7\" (UID: \"3ecd8a36-969e-4f19-837b-e4c164de1b50\") " pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.424756 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.424734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-586xd\" (UniqueName: \"kubernetes.io/projected/3ecd8a36-969e-4f19-837b-e4c164de1b50-kube-api-access-586xd\") pod \"seaweedfs-64568bcd49-lrtg7\" (UID: \"3ecd8a36-969e-4f19-837b-e4c164de1b50\") " pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.425682 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.425656    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:13.525794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.525759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-586xd\" (UniqueName: \"kubernetes.io/projected/3ecd8a36-969e-4f19-837b-e4c164de1b50-kube-api-access-586xd\") pod \"seaweedfs-64568bcd49-lrtg7\" (UID: \"3ecd8a36-969e-4f19-837b-e4c164de1b50\") " pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.525794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.525806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3ecd8a36-969e-4f19-837b-e4c164de1b50-data\") pod \"seaweedfs-64568bcd49-lrtg7\" (UID: \"3ecd8a36-969e-4f19-837b-e4c164de1b50\") " pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.526215 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.526167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3ecd8a36-969e-4f19-837b-e4c164de1b50-data\") pod \"seaweedfs-64568bcd49-lrtg7\" (UID: \"3ecd8a36-969e-4f19-837b-e4c164de1b50\") " pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.537140 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.537114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-586xd\" (UniqueName: \"kubernetes.io/projected/3ecd8a36-969e-4f19-837b-e4c164de1b50-kube-api-access-586xd\") pod \"seaweedfs-64568bcd49-lrtg7\" (UID: \"3ecd8a36-969e-4f19-837b-e4c164de1b50\") " pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.540219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.540160    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:13.662525 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.662436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:13.803972 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.803951    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-lrtg7"]
Jul 09 18:10:13.806534 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:10:13.806506    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecd8a36_969e_4f19_837b_e4c164de1b50.slice/crio-b5e75aa8a474f289a85aa8101291518ea4b92385ca957d73d5e0ebf183b40edb WatchSource:0}: Error finding container b5e75aa8a474f289a85aa8101291518ea4b92385ca957d73d5e0ebf183b40edb: Status 404 returned error can't find the container with id b5e75aa8a474f289a85aa8101291518ea4b92385ca957d73d5e0ebf183b40edb
Jul 09 18:10:13.807679 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:13.807658    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:10:14.553014 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:14.552977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-lrtg7" event={"ID":"3ecd8a36-969e-4f19-837b-e4c164de1b50","Type":"ContainerStarted","Data":"b5e75aa8a474f289a85aa8101291518ea4b92385ca957d73d5e0ebf183b40edb"}
Jul 09 18:10:14.875500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:14.875463    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:14.972611 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:14.972578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:19.575325 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:19.575289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-lrtg7" event={"ID":"3ecd8a36-969e-4f19-837b-e4c164de1b50","Type":"ContainerStarted","Data":"947c95386298525a657927d4ba41204255033d89eeeb3dc2746f8e270130b7d5"}
Jul 09 18:10:19.575714 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:19.575417    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:19.594915 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:19.594870    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-lrtg7" podStartSLOduration=1.679508365 podStartE2EDuration="6.594854618s" totalImagesPullingTime="4.915346253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:10:19.593599811 +0000 UTC m=+413.517144283" watchObservedRunningTime="2026-07-09 18:10:19.594854618 +0000 UTC m=+413.518399090"
Jul 09 18:10:25.582147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:25.582111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-lrtg7"
Jul 09 18:10:52.492953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:52.492850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:10:52.591741 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:10:52.591706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:04.060961 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:04.060928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:04.164929 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:04.164894    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:08.125636 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:08.125586    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:08.299279 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:08.299240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:09.694832 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:09.694795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:09.873864 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:09.873831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:10.016745 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:10.016665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:10.171627 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:10.171590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:10.322231 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:10.322105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:12.357900 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.357866    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:12.714819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.714724    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:12.836847 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.836816    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"]
Jul 09 18:11:12.840021 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.840006    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:12.848622 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.848602    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 18:11:12.849158 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.849144    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 18:11:12.852070 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.852052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-c5b85\""
Jul 09 18:11:12.868321 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.868295    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"]
Jul 09 18:11:12.932708 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.932674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5221c804-c09d-496e-910a-494e1fc851c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5nkct\" (UID: \"5221c804-c09d-496e-910a-494e1fc851c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:12.932708 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:12.932709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg64r\" (UniqueName: \"kubernetes.io/projected/5221c804-c09d-496e-910a-494e1fc851c0-kube-api-access-wg64r\") pod \"cert-manager-cainjector-559cdd8c87-5nkct\" (UID: \"5221c804-c09d-496e-910a-494e1fc851c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:13.033640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.033560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5221c804-c09d-496e-910a-494e1fc851c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5nkct\" (UID: \"5221c804-c09d-496e-910a-494e1fc851c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:13.033640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.033590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg64r\" (UniqueName: \"kubernetes.io/projected/5221c804-c09d-496e-910a-494e1fc851c0-kube-api-access-wg64r\") pod \"cert-manager-cainjector-559cdd8c87-5nkct\" (UID: \"5221c804-c09d-496e-910a-494e1fc851c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:13.048119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.048097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg64r\" (UniqueName: \"kubernetes.io/projected/5221c804-c09d-496e-910a-494e1fc851c0-kube-api-access-wg64r\") pod \"cert-manager-cainjector-559cdd8c87-5nkct\" (UID: \"5221c804-c09d-496e-910a-494e1fc851c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:13.048730 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.048712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5221c804-c09d-496e-910a-494e1fc851c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-5nkct\" (UID: \"5221c804-c09d-496e-910a-494e1fc851c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:13.168910 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.168883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"
Jul 09 18:11:13.317592 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.317543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-5nkct"]
Jul 09 18:11:13.320321 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:11:13.320294    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5221c804_c09d_496e_910a_494e1fc851c0.slice/crio-a6eecb3f6e1b1ba0273a19bfa806e4561c8943a60d61ace1a61983badb02e5b7 WatchSource:0}: Error finding container a6eecb3f6e1b1ba0273a19bfa806e4561c8943a60d61ace1a61983badb02e5b7: Status 404 returned error can't find the container with id a6eecb3f6e1b1ba0273a19bfa806e4561c8943a60d61ace1a61983badb02e5b7
Jul 09 18:11:13.705920 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.705884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:13.792069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.792035    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct" event={"ID":"5221c804-c09d-496e-910a-494e1fc851c0","Type":"ContainerStarted","Data":"a6eecb3f6e1b1ba0273a19bfa806e4561c8943a60d61ace1a61983badb02e5b7"}
Jul 09 18:11:13.847567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:13.847532    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:14.075877 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:14.075773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:14.370436 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:14.370401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:17.811910 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:17.811819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct" event={"ID":"5221c804-c09d-496e-910a-494e1fc851c0","Type":"ContainerStarted","Data":"615e69a9b47cc3ab23534f6a0fd212bf521ee82bccf1f8735b375ab8d67b4116"}
Jul 09 18:11:17.841304 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:17.841257    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-5nkct" podStartSLOduration=1.808214661 podStartE2EDuration="5.841236721s" totalImagesPullingTime="4.03302206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:17.839657087 +0000 UTC m=+471.763201559" watchObservedRunningTime="2026-07-09 18:11:17.841236721 +0000 UTC m=+471.764781196"
Jul 09 18:11:18.174389 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.174352    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nx49g"]
Jul 09 18:11:18.177762 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.177746    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.180454 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.180428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hwpts\""
Jul 09 18:11:18.190826 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.190806    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nx49g"]
Jul 09 18:11:18.282628 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.282600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d630cfb8-f397-4cc5-8df3-93f0759ba0d3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nx49g\" (UID: \"d630cfb8-f397-4cc5-8df3-93f0759ba0d3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.282789 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.282661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msqkx\" (UniqueName: \"kubernetes.io/projected/d630cfb8-f397-4cc5-8df3-93f0759ba0d3-kube-api-access-msqkx\") pod \"cert-manager-webhook-6486ffd997-nx49g\" (UID: \"d630cfb8-f397-4cc5-8df3-93f0759ba0d3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.383451 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.383422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d630cfb8-f397-4cc5-8df3-93f0759ba0d3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nx49g\" (UID: \"d630cfb8-f397-4cc5-8df3-93f0759ba0d3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.383620 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.383474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msqkx\" (UniqueName: \"kubernetes.io/projected/d630cfb8-f397-4cc5-8df3-93f0759ba0d3-kube-api-access-msqkx\") pod \"cert-manager-webhook-6486ffd997-nx49g\" (UID: \"d630cfb8-f397-4cc5-8df3-93f0759ba0d3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.392931 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.392906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d630cfb8-f397-4cc5-8df3-93f0759ba0d3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nx49g\" (UID: \"d630cfb8-f397-4cc5-8df3-93f0759ba0d3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.393113 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.393090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msqkx\" (UniqueName: \"kubernetes.io/projected/d630cfb8-f397-4cc5-8df3-93f0759ba0d3-kube-api-access-msqkx\") pod \"cert-manager-webhook-6486ffd997-nx49g\" (UID: \"d630cfb8-f397-4cc5-8df3-93f0759ba0d3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.487992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.487907    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.617795 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.617771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nx49g"]
Jul 09 18:11:18.620387 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:11:18.620362    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd630cfb8_f397_4cc5_8df3_93f0759ba0d3.slice/crio-34a92c182637eacb1fd5129dfe5cc4c9101eb2a897e1af7c52989b2971f2ad01 WatchSource:0}: Error finding container 34a92c182637eacb1fd5129dfe5cc4c9101eb2a897e1af7c52989b2971f2ad01: Status 404 returned error can't find the container with id 34a92c182637eacb1fd5129dfe5cc4c9101eb2a897e1af7c52989b2971f2ad01
Jul 09 18:11:18.817851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.817753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g" event={"ID":"d630cfb8-f397-4cc5-8df3-93f0759ba0d3","Type":"ContainerStarted","Data":"205af7f29b0670a4ee667bb894178c3e02522d86e0ad971b450be86b1014b4a8"}
Jul 09 18:11:18.817851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.817808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g" event={"ID":"d630cfb8-f397-4cc5-8df3-93f0759ba0d3","Type":"ContainerStarted","Data":"34a92c182637eacb1fd5129dfe5cc4c9101eb2a897e1af7c52989b2971f2ad01"}
Jul 09 18:11:18.818367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.817921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:18.848099 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.848048    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g" podStartSLOduration=0.848030787 podStartE2EDuration="848.030787ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:18.847233966 +0000 UTC m=+472.770778435" watchObservedRunningTime="2026-07-09 18:11:18.848030787 +0000 UTC m=+472.771575259"
Jul 09 18:11:18.910696 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:18.910661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:19.011018 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:19.010981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:20.034869 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:20.034835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:20.853662 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:20.853627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:20.963529 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:20.963490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:21.098716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.098677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:21.519447 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.519415    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zxqjb"]
Jul 09 18:11:21.522538 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.522520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.525208 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.525178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7fghq\""
Jul 09 18:11:21.535891 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.535866    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zxqjb"]
Jul 09 18:11:21.710483 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.710451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/49cd373f-7149-4061-bba5-2ae7b07f1319-bound-sa-token\") pod \"cert-manager-55b4d954d5-zxqjb\" (UID: \"49cd373f-7149-4061-bba5-2ae7b07f1319\") " pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.710644 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.710514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f87lk\" (UniqueName: \"kubernetes.io/projected/49cd373f-7149-4061-bba5-2ae7b07f1319-kube-api-access-f87lk\") pod \"cert-manager-55b4d954d5-zxqjb\" (UID: \"49cd373f-7149-4061-bba5-2ae7b07f1319\") " pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.811017 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.810928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/49cd373f-7149-4061-bba5-2ae7b07f1319-bound-sa-token\") pod \"cert-manager-55b4d954d5-zxqjb\" (UID: \"49cd373f-7149-4061-bba5-2ae7b07f1319\") " pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.811017 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.810993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f87lk\" (UniqueName: \"kubernetes.io/projected/49cd373f-7149-4061-bba5-2ae7b07f1319-kube-api-access-f87lk\") pod \"cert-manager-55b4d954d5-zxqjb\" (UID: \"49cd373f-7149-4061-bba5-2ae7b07f1319\") " pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.819978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.819949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/49cd373f-7149-4061-bba5-2ae7b07f1319-bound-sa-token\") pod \"cert-manager-55b4d954d5-zxqjb\" (UID: \"49cd373f-7149-4061-bba5-2ae7b07f1319\") " pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.820970 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.820949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f87lk\" (UniqueName: \"kubernetes.io/projected/49cd373f-7149-4061-bba5-2ae7b07f1319-kube-api-access-f87lk\") pod \"cert-manager-55b4d954d5-zxqjb\" (UID: \"49cd373f-7149-4061-bba5-2ae7b07f1319\") " pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.831485 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.831451    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zxqjb"
Jul 09 18:11:21.959980 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:21.959838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zxqjb"]
Jul 09 18:11:21.962140 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:11:21.962117    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49cd373f_7149_4061_bba5_2ae7b07f1319.slice/crio-79f8a7416533ec689de94cda90af31d6843628d04b424af64642bc3353db97ec WatchSource:0}: Error finding container 79f8a7416533ec689de94cda90af31d6843628d04b424af64642bc3353db97ec: Status 404 returned error can't find the container with id 79f8a7416533ec689de94cda90af31d6843628d04b424af64642bc3353db97ec
Jul 09 18:11:22.133338 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:22.133306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:22.839370 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:22.839334    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zxqjb" event={"ID":"49cd373f-7149-4061-bba5-2ae7b07f1319","Type":"ContainerStarted","Data":"ca14c2552b5a0a83ae739a5aaede84d602ec6397b3483157009e273f4393c133"}
Jul 09 18:11:22.839516 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:22.839379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zxqjb" event={"ID":"49cd373f-7149-4061-bba5-2ae7b07f1319","Type":"ContainerStarted","Data":"79f8a7416533ec689de94cda90af31d6843628d04b424af64642bc3353db97ec"}
Jul 09 18:11:22.857564 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:22.857503    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-zxqjb" podStartSLOduration=1.857480348 podStartE2EDuration="1.857480348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:22.856493868 +0000 UTC m=+476.780038343" watchObservedRunningTime="2026-07-09 18:11:22.857480348 +0000 UTC m=+476.781024821"
Jul 09 18:11:24.824011 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:24.823980    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nx49g"
Jul 09 18:11:24.835039 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:24.835008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:24.933009 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:24.932975    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:37.594513 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:37.594475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:43.163485 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:43.163452    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:43.284169 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:43.284139    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:44.659020 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:44.658984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:52.345678 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.345637    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:52.407047 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.407018    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"]
Jul 09 18:11:52.409177 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.409161    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.412794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.412772    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 18:11:52.414009 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.413986    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 18:11:52.414178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.414160    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-j5mvz\""
Jul 09 18:11:52.414291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.414076    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 18:11:52.414291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.414135    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 18:11:52.414291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.414090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:11:52.422336 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.422316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"]
Jul 09 18:11:52.467651 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.467619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:52.478706 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.478681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5c11113e-3839-4563-ab30-b90de9b2247e-manager-config\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.478816 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.478723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c11113e-3839-4563-ab30-b90de9b2247e-cert\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.478816 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.478778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c11113e-3839-4563-ab30-b90de9b2247e-metrics-cert\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.478886 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.478848    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj4l2\" (UniqueName: \"kubernetes.io/projected/5c11113e-3839-4563-ab30-b90de9b2247e-kube-api-access-zj4l2\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.579621 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.579585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj4l2\" (UniqueName: \"kubernetes.io/projected/5c11113e-3839-4563-ab30-b90de9b2247e-kube-api-access-zj4l2\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.579783 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.579643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5c11113e-3839-4563-ab30-b90de9b2247e-manager-config\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.579783 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.579676    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c11113e-3839-4563-ab30-b90de9b2247e-cert\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.579783 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.579699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c11113e-3839-4563-ab30-b90de9b2247e-metrics-cert\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.580443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.580420    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5c11113e-3839-4563-ab30-b90de9b2247e-manager-config\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.582335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.582313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5c11113e-3839-4563-ab30-b90de9b2247e-cert\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.582423 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.582323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c11113e-3839-4563-ab30-b90de9b2247e-metrics-cert\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.592987 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.592964    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj4l2\" (UniqueName: \"kubernetes.io/projected/5c11113e-3839-4563-ab30-b90de9b2247e-kube-api-access-zj4l2\") pod \"lws-controller-manager-7885457d66-b8k5x\" (UID: \"5c11113e-3839-4563-ab30-b90de9b2247e\") " pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.718806 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.718768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:52.857402 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.857359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"]
Jul 09 18:11:52.962896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:52.962862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x" event={"ID":"5c11113e-3839-4563-ab30-b90de9b2247e","Type":"ContainerStarted","Data":"c5f39f72127e1037ee8b2827e7447072bc33a59003c0405ba2995a33858052f2"}
Jul 09 18:11:55.978670 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:55.978637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x" event={"ID":"5c11113e-3839-4563-ab30-b90de9b2247e","Type":"ContainerStarted","Data":"2cdd4eef7b8850e9766a40dc0b0cabb7912b781f15456a6cf3303fb22d75a618"}
Jul 09 18:11:55.979087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:55.978692    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:11:56.020431 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:56.020380    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x" podStartSLOduration=1.5583678010000002 podStartE2EDuration="4.020364126s" totalImagesPullingTime="2.461996325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:56.019375119 +0000 UTC m=+509.942919593" watchObservedRunningTime="2026-07-09 18:11:56.020364126 +0000 UTC m=+509.943908597"
Jul 09 18:11:58.737371 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:58.737335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:11:58.839548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:11:58.839507    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:02.446345 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:02.446294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:02.542957 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:02.542929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:05.754526 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:05.754494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:06.985275 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:06.985247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7885457d66-b8k5x"
Jul 09 18:12:40.258034 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:40.257991    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:40.360734 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:40.360700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:52.725858 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:52.725826    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:53.332598 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:53.332555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:56.031765 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:56.031729    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.440492 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.440454    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"]
Jul 09 18:12:57.447993 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.447962    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:12:57.452974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.452951    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 18:12:57.453097 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.452953    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 18:12:57.453097 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.453008    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-b44cq\""
Jul 09 18:12:57.454005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.453982    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 18:12:57.456837 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.456816    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"]
Jul 09 18:12:57.542252 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.542216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94csf\" (UniqueName: \"kubernetes.io/projected/5672237d-5983-4c0c-874c-10bcf5c09cad-kube-api-access-94csf\") pod \"dns-operator-controller-manager-65b49595d7-94pfn\" (UID: \"5672237d-5983-4c0c-874c-10bcf5c09cad\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:12:57.628039 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.628007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:57.643356 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.643331    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94csf\" (UniqueName: \"kubernetes.io/projected/5672237d-5983-4c0c-874c-10bcf5c09cad-kube-api-access-94csf\") pod \"dns-operator-controller-manager-65b49595d7-94pfn\" (UID: \"5672237d-5983-4c0c-874c-10bcf5c09cad\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:12:57.658112 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.658085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94csf\" (UniqueName: \"kubernetes.io/projected/5672237d-5983-4c0c-874c-10bcf5c09cad-kube-api-access-94csf\") pod \"dns-operator-controller-manager-65b49595d7-94pfn\" (UID: \"5672237d-5983-4c0c-874c-10bcf5c09cad\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:12:57.759777 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.759698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:12:57.895101 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:57.895073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"]
Jul 09 18:12:57.897048 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:12:57.897008    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5672237d_5983_4c0c_874c_10bcf5c09cad.slice/crio-506ca1b2472637975c6f430d3182546e0b556315a921a4e37558f7b54b9456ed WatchSource:0}: Error finding container 506ca1b2472637975c6f430d3182546e0b556315a921a4e37558f7b54b9456ed: Status 404 returned error can't find the container with id 506ca1b2472637975c6f430d3182546e0b556315a921a4e37558f7b54b9456ed
Jul 09 18:12:58.014180 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:58.014101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:58.255995 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:58.255958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn" event={"ID":"5672237d-5983-4c0c-874c-10bcf5c09cad","Type":"ContainerStarted","Data":"506ca1b2472637975c6f430d3182546e0b556315a921a4e37558f7b54b9456ed"}
Jul 09 18:12:58.330441 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:58.330355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:12:59.773865 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:12:59.773819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:00.242363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.242278    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-trkjt"]
Jul 09 18:13:00.249004 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.248979    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:00.253879 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.253856    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vms7k\""
Jul 09 18:13:00.264161 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.264129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sdtl\" (UniqueName: \"kubernetes.io/projected/14223905-4d68-4886-9f9f-e1dff711bac0-kube-api-access-5sdtl\") pod \"authorino-operator-6d85f6564-trkjt\" (UID: \"14223905-4d68-4886-9f9f-e1dff711bac0\") " pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:00.268365 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.268342    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-trkjt"]
Jul 09 18:13:00.365676 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.365638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sdtl\" (UniqueName: \"kubernetes.io/projected/14223905-4d68-4886-9f9f-e1dff711bac0-kube-api-access-5sdtl\") pod \"authorino-operator-6d85f6564-trkjt\" (UID: \"14223905-4d68-4886-9f9f-e1dff711bac0\") " pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:00.386084 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.386047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sdtl\" (UniqueName: \"kubernetes.io/projected/14223905-4d68-4886-9f9f-e1dff711bac0-kube-api-access-5sdtl\") pod \"authorino-operator-6d85f6564-trkjt\" (UID: \"14223905-4d68-4886-9f9f-e1dff711bac0\") " pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:00.560813 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.560728    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:00.629147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.629101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:00.981544 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:00.981499    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-trkjt"]
Jul 09 18:13:00.983880 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:13:00.983853    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14223905_4d68_4886_9f9f_e1dff711bac0.slice/crio-a063f32fb8869df000783a2661a24bac698269ed0376b92248d772abeaebb906 WatchSource:0}: Error finding container a063f32fb8869df000783a2661a24bac698269ed0376b92248d772abeaebb906: Status 404 returned error can't find the container with id a063f32fb8869df000783a2661a24bac698269ed0376b92248d772abeaebb906
Jul 09 18:13:01.104701 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.104664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:01.275066 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.275030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn" event={"ID":"5672237d-5983-4c0c-874c-10bcf5c09cad","Type":"ContainerStarted","Data":"cc7ce3dc16776f9f76a49715dd143e099c38f61ef2852f268ec130bf3804bf81"}
Jul 09 18:13:01.275277 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.275146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:13:01.276362 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.276338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt" event={"ID":"14223905-4d68-4886-9f9f-e1dff711bac0","Type":"ContainerStarted","Data":"a063f32fb8869df000783a2661a24bac698269ed0376b92248d772abeaebb906"}
Jul 09 18:13:01.338419 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.338368    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn" podStartSLOduration=1.376761791 podStartE2EDuration="4.338351532s" totalImagesPullingTime="2.961589741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:13:01.331918294 +0000 UTC m=+575.255462765" watchObservedRunningTime="2026-07-09 18:13:01.338351532 +0000 UTC m=+575.261896056"
Jul 09 18:13:01.474939 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.474854    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:01.790545 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.790464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:01.972943 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:01.972903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:03.293217 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:03.293112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt" event={"ID":"14223905-4d68-4886-9f9f-e1dff711bac0","Type":"ContainerStarted","Data":"cbd2eb501a803ea535e46e3d0d2f50bf4e6bf05740ce8eb5326a6fa5fbfadc12"}
Jul 09 18:13:03.293566 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:03.293250    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:03.335195 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:03.335130    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt" podStartSLOduration=1.315107372 podStartE2EDuration="3.335112691s" totalImagesPullingTime="2.020005319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:13:03.333057633 +0000 UTC m=+577.256602105" watchObservedRunningTime="2026-07-09 18:13:03.335112691 +0000 UTC m=+577.258657163"
Jul 09 18:13:03.381513 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:03.381462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:03.495634 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:03.495602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:05.838702 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:05.838671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:05.951608 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:05.951575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:08.840938 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:08.840902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:08.942208 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:08.942156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:11.463440 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.463403    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"]
Jul 09 18:13:11.465831 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.465815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.470373 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.470346    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 09 18:13:11.470501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.470442    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 09 18:13:11.470501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.470455    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gv6vk\""
Jul 09 18:13:11.474763 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.474738    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:11.487948 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.487911    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"]
Jul 09 18:13:11.562021 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.561988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnw68\" (UniqueName: \"kubernetes.io/projected/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-kube-api-access-dnw68\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.562247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.562043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.562247 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.562127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.663103 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.663068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnw68\" (UniqueName: \"kubernetes.io/projected/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-kube-api-access-dnw68\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.663308 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.663118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.663308 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.663171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.663308 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:13:11.663300    2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 09 18:13:11.663414 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:13:11.663385    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-plugin-serving-cert podName:deb31e7c-4910-4fb8-ab8f-a2a61cc0560b nodeName:}" failed. No retries permitted until 2026-07-09 18:13:12.163367152 +0000 UTC m=+586.086911604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-plugin-serving-cert") pod "kuadrant-console-plugin-85fd4b8b9d-r9kdf" (UID: "deb31e7c-4910-4fb8-ab8f-a2a61cc0560b") : secret "plugin-serving-cert" not found
Jul 09 18:13:11.663770 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.663750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.674963 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.674939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnw68\" (UniqueName: \"kubernetes.io/projected/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-kube-api-access-dnw68\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:11.744803 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:11.744714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:12.168283 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:12.168246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:12.170605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:12.170583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/deb31e7c-4910-4fb8-ab8f-a2a61cc0560b-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-r9kdf\" (UID: \"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:12.288304 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:12.288275    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-94pfn"
Jul 09 18:13:12.374636 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:12.374606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"
Jul 09 18:13:12.519288 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:12.519254    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf"]
Jul 09 18:13:12.521439 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:13:12.521412    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddeb31e7c_4910_4fb8_ab8f_a2a61cc0560b.slice/crio-26ab25fd3505cb950219d9d16a4bbd0c7f3cbcb509e8c91b211523e233718ea4 WatchSource:0}: Error finding container 26ab25fd3505cb950219d9d16a4bbd0c7f3cbcb509e8c91b211523e233718ea4: Status 404 returned error can't find the container with id 26ab25fd3505cb950219d9d16a4bbd0c7f3cbcb509e8c91b211523e233718ea4
Jul 09 18:13:13.344394 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:13.344347    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf" event={"ID":"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b","Type":"ContainerStarted","Data":"26ab25fd3505cb950219d9d16a4bbd0c7f3cbcb509e8c91b211523e233718ea4"}
Jul 09 18:13:14.300647 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:14.300619    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6d85f6564-trkjt"
Jul 09 18:13:14.441319 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:14.441286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:14.542318 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:14.542274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:17.375407 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:17.375367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf" event={"ID":"deb31e7c-4910-4fb8-ab8f-a2a61cc0560b","Type":"ContainerStarted","Data":"bfbc15dbfc7ea2e8f009f0fc5f892732642601754ff1380f17ab1bc28e109906"}
Jul 09 18:13:17.396926 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:17.396881    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-r9kdf" podStartSLOduration=2.027186191 podStartE2EDuration="6.396862546s" totalImagesPullingTime="4.369676355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:13:17.395665217 +0000 UTC m=+591.319209690" watchObservedRunningTime="2026-07-09 18:13:17.396862546 +0000 UTC m=+591.320407018"
Jul 09 18:13:19.249886 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:19.249852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:19.349049 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:19.349018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:19.454947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:19.454908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:23.870777 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:23.870734    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:25.919738 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:25.919699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:26.019008 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:26.018977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:26.526929 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:26.526901    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:13:26.534636 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:26.534613    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:13:28.901173 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:28.901135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:29.002056 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:29.002016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:34.700864 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:34.700827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:34.801705 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:34.801667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:35.270355 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:35.270325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:35.376178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:35.376146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:38.772387 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:38.772356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:38.871892 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:38.871859    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:38.981562 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:38.981529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:53.038824 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.038788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:53.159471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.159431    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:53.286164 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.286124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:53.484818 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.484789    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-xw8v8"]
Jul 09 18:13:53.488078 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.488061    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.491667 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.491640    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 18:13:53.516822 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.516801    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-xw8v8"]
Jul 09 18:13:53.541756 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.541734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtjnq\" (UniqueName: \"kubernetes.io/projected/d449c6ba-4429-4bed-a25d-09af63bc817b-kube-api-access-jtjnq\") pod \"limitador-limitador-69574b596d-xw8v8\" (UID: \"d449c6ba-4429-4bed-a25d-09af63bc817b\") " pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.541861 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.541782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d449c6ba-4429-4bed-a25d-09af63bc817b-config-file\") pod \"limitador-limitador-69574b596d-xw8v8\" (UID: \"d449c6ba-4429-4bed-a25d-09af63bc817b\") " pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.582622 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.582592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-xw8v8"]
Jul 09 18:13:53.643271 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.643244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d449c6ba-4429-4bed-a25d-09af63bc817b-config-file\") pod \"limitador-limitador-69574b596d-xw8v8\" (UID: \"d449c6ba-4429-4bed-a25d-09af63bc817b\") " pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.643446 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.643423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtjnq\" (UniqueName: \"kubernetes.io/projected/d449c6ba-4429-4bed-a25d-09af63bc817b-kube-api-access-jtjnq\") pod \"limitador-limitador-69574b596d-xw8v8\" (UID: \"d449c6ba-4429-4bed-a25d-09af63bc817b\") " pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.643919 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.643903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d449c6ba-4429-4bed-a25d-09af63bc817b-config-file\") pod \"limitador-limitador-69574b596d-xw8v8\" (UID: \"d449c6ba-4429-4bed-a25d-09af63bc817b\") " pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.661669 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.661636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtjnq\" (UniqueName: \"kubernetes.io/projected/d449c6ba-4429-4bed-a25d-09af63bc817b-kube-api-access-jtjnq\") pod \"limitador-limitador-69574b596d-xw8v8\" (UID: \"d449c6ba-4429-4bed-a25d-09af63bc817b\") " pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.797942 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.797853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:53.946410 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:53.946387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-xw8v8"]
Jul 09 18:13:53.948273 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:13:53.948245    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd449c6ba_4429_4bed_a25d_09af63bc817b.slice/crio-37bf99694b50c0f619af1c9e29e0ac45b801dec803fe8b0a2c2e753eaec6298a WatchSource:0}: Error finding container 37bf99694b50c0f619af1c9e29e0ac45b801dec803fe8b0a2c2e753eaec6298a: Status 404 returned error can't find the container with id 37bf99694b50c0f619af1c9e29e0ac45b801dec803fe8b0a2c2e753eaec6298a
Jul 09 18:13:54.123054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:54.123019    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:54.291590 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:54.291542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:13:54.551337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:54.551203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8" event={"ID":"d449c6ba-4429-4bed-a25d-09af63bc817b","Type":"ContainerStarted","Data":"37bf99694b50c0f619af1c9e29e0ac45b801dec803fe8b0a2c2e753eaec6298a"}
Jul 09 18:13:58.573521 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:58.573485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8" event={"ID":"d449c6ba-4429-4bed-a25d-09af63bc817b","Type":"ContainerStarted","Data":"ad1946c724f36a92118faf55b39e79528c204d1a557098e16a9df79c2d245136"}
Jul 09 18:13:58.573981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:58.573603    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:13:58.593410 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:13:58.593230    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8" podStartSLOduration=1.932079566 podStartE2EDuration="5.593215507s" totalImagesPullingTime="3.661135941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:13:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:13:58.592585911 +0000 UTC m=+632.516130383" watchObservedRunningTime="2026-07-09 18:13:58.593215507 +0000 UTC m=+632.516759977"
Jul 09 18:14:09.579008 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:14:09.578976    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-69574b596d-xw8v8"
Jul 09 18:14:15.754898 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:14:15.754855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:17.209219 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:14:17.209145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:36.758007 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:14:36.757965    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:36.862784 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:14:36.862742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:14:51.630596 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:14:51.630561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:00.663511 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:15:00.663471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:00.746291 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:15:00.746251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:00.842072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:15:00.842037    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:00.940432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:15:00.940334    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:01.504955 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:15:01.504915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:15:01.605158 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:15:01.605127    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:37.254437 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:16:37.254399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:38.764392 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:16:38.764362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:50.083356 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:16:50.083325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:50.183708 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:16:50.183675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:54.778432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:16:54.778390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:16:54.878925 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:16:54.878893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:00.797838 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:00.797803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:29.279958 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:29.279924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:29.701142 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:29.701110    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:31.887349 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:31.887319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:41.895794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.895758    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"]
Jul 09 18:17:41.899467 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.899443    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:41.904580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.904555    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:17:41.904580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.904558    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Jul 09 18:17:41.904766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.904718    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:17:41.904829 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.904794    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:17:41.914956 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.914931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"]
Jul 09 18:17:41.933557 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.933528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:41.964015 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.963982    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:41.964167 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.964016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znlfm\" (UniqueName: \"kubernetes.io/projected/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-kube-api-access-znlfm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:41.964167 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.964048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:41.964167 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.964095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:41.964353 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.964175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:41.964353 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:41.964263    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065043 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065258 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znlfm\" (UniqueName: \"kubernetes.io/projected/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-kube-api-access-znlfm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065258 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065258 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065258 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065258 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065525 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065584 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.065640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.065591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.067278 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.067253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.067455 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.067438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.075559 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.075539    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znlfm\" (UniqueName: \"kubernetes.io/projected/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-kube-api-access-znlfm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.209894 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.209817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:42.343102 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.343078    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"]
Jul 09 18:17:42.345519 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:17:42.345485    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60abb6cc_ab5b_4508_ba3f_d8fc17cd0cd8.slice/crio-6d75c8d7d06f6e3351dc603701bcf6e6e4aac2be4c52b23b006acc732663a756 WatchSource:0}: Error finding container 6d75c8d7d06f6e3351dc603701bcf6e6e4aac2be4c52b23b006acc732663a756: Status 404 returned error can't find the container with id 6d75c8d7d06f6e3351dc603701bcf6e6e4aac2be4c52b23b006acc732663a756
Jul 09 18:17:42.347444 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.347428    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:17:42.639771 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:42.639729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn" event={"ID":"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8","Type":"ContainerStarted","Data":"6d75c8d7d06f6e3351dc603701bcf6e6e4aac2be4c52b23b006acc732663a756"}
Jul 09 18:17:43.646327 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:43.646295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn" event={"ID":"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8","Type":"ContainerStarted","Data":"fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f"}
Jul 09 18:17:43.664323 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:43.664274    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn" podStartSLOduration=1.591250311 podStartE2EDuration="2.664258725s" totalImagesPullingTime="1.073008414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:17:43.662479531 +0000 UTC m=+857.586024002" watchObservedRunningTime="2026-07-09 18:17:43.664258725 +0000 UTC m=+857.587803197"
Jul 09 18:17:44.210467 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:44.210432    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:52.210283 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:52.210244    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:52.223027 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:52.222997    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:52.700303 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:52.700268    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:17:57.106138 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:57.106101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:17:59.001788 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:59.001755    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"]
Jul 09 18:17:59.029287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:17:59.029255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:14.022234 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.022163    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn" podUID="60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" containerName="main" containerID="cri-o://fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f" gracePeriod=45
Jul 09 18:18:14.268460 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.268437    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:18:14.343650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343567    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-dshm\") pod \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") "
Jul 09 18:18:14.343650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-model-cache\") pod \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") "
Jul 09 18:18:14.343874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343657    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znlfm\" (UniqueName: \"kubernetes.io/projected/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-kube-api-access-znlfm\") pod \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") "
Jul 09 18:18:14.343874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343681    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tls-certs\") pod \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") "
Jul 09 18:18:14.343874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343719    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tmp-dir\") pod \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") "
Jul 09 18:18:14.343874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343839    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-home\") pod \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\" (UID: \"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8\") "
Jul 09 18:18:14.344070 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343934    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" (UID: "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:18:14.344070 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.343951    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" (UID: "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:18:14.344144 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.344064    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-home" (OuterVolumeSpecName: "home") pod "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" (UID: "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:18:14.344252 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.344231    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:18:14.344306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.344258    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:18:14.344306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.344273    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:18:14.345852 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.345833    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-dshm" (OuterVolumeSpecName: "dshm") pod "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" (UID: "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:18:14.345923 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.345881    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" (UID: "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:18:14.345923 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.345883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-kube-api-access-znlfm" (OuterVolumeSpecName: "kube-api-access-znlfm") pod "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" (UID: "60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8"). InnerVolumeSpecName "kube-api-access-znlfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:18:14.445519 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.445482    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znlfm\" (UniqueName: \"kubernetes.io/projected/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-kube-api-access-znlfm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:18:14.445519 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.445513    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:18:14.445696 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.445528    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:18:14.799615 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.799581    2581 generic.go:356] "Generic (PLEG): container finished" podID="60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" containerID="fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f" exitCode=0
Jul 09 18:18:14.799804 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.799693    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"
Jul 09 18:18:14.799804 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.799692    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn" event={"ID":"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8","Type":"ContainerDied","Data":"fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f"}
Jul 09 18:18:14.799804 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.799733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn" event={"ID":"60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8","Type":"ContainerDied","Data":"6d75c8d7d06f6e3351dc603701bcf6e6e4aac2be4c52b23b006acc732663a756"}
Jul 09 18:18:14.799804 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.799756    2581 scope.go:117] "RemoveContainer" containerID="fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f"
Jul 09 18:18:14.809110 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.809091    2581 scope.go:117] "RemoveContainer" containerID="fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f"
Jul 09 18:18:14.809408 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:18:14.809384    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f\": container with ID starting with fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f not found: ID does not exist" containerID="fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f"
Jul 09 18:18:14.809492 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.809419    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f"} err="failed to get container status \"fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f\": rpc error: code = NotFound desc = could not find container \"fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f\": container with ID starting with fd90fc4bc5e6b4ea3c6d3523e194bbcb681c5ac237757affc637d1ead304099f not found: ID does not exist"
Jul 09 18:18:14.827058 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.827023    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"]
Jul 09 18:18:14.839904 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.839884    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b87nbn"]
Jul 09 18:18:14.937478 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:14.937449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:15.141750 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.141722    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"]
Jul 09 18:18:15.142116 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.142100    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" containerName="main"
Jul 09 18:18:15.142116 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.142112    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" containerName="main"
Jul 09 18:18:15.142254 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.142242    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" containerName="main"
Jul 09 18:18:15.145270 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.145249    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.147867 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.147844    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:18:15.149027 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.148999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-enabled-test-kserve-self-signed-certs\""
Jul 09 18:18:15.149207 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.149037    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:18:15.149207 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.149048    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:18:15.149207 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.149148    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:18:15.149442 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.149318    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-enabled-test-epp-sa-dockercfg-drr8l\""
Jul 09 18:18:15.157346 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.157328    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"]
Jul 09 18:18:15.175273 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.175251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:15.252418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.252387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-kserve-provision-location\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.252555 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.252423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhdwr\" (UniqueName: \"kubernetes.io/projected/818db810-07e6-457a-8bef-ca361d746c2a-kube-api-access-jhdwr\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.252555 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.252463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818db810-07e6-457a-8bef-ca361d746c2a-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.252555 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.252524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-tokenizer-uds\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.353897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.353866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818db810-07e6-457a-8bef-ca361d746c2a-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.354072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.353907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-tokenizer-uds\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.354072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.353955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-kserve-provision-location\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.354072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.353978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhdwr\" (UniqueName: \"kubernetes.io/projected/818db810-07e6-457a-8bef-ca361d746c2a-kube-api-access-jhdwr\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.354410 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.354387    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-kserve-provision-location\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.354480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.354420    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-tokenizer-uds\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.356543 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.356521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818db810-07e6-457a-8bef-ca361d746c2a-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.366117 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.366093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhdwr\" (UniqueName: \"kubernetes.io/projected/818db810-07e6-457a-8bef-ca361d746c2a-kube-api-access-jhdwr\") pod \"auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") " pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.454568 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.454487    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:15.586352 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.586208    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"]
Jul 09 18:18:15.589114 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:18:15.589090    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818db810_07e6_457a_8bef_ca361d746c2a.slice/crio-57eb0db268e56884ab53c55e51987fe93d32e33dce5e4ee654756528fd45e6ee WatchSource:0}: Error finding container 57eb0db268e56884ab53c55e51987fe93d32e33dce5e4ee654756528fd45e6ee: Status 404 returned error can't find the container with id 57eb0db268e56884ab53c55e51987fe93d32e33dce5e4ee654756528fd45e6ee
Jul 09 18:18:15.805842 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:15.805755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" event={"ID":"818db810-07e6-457a-8bef-ca361d746c2a","Type":"ContainerStarted","Data":"57eb0db268e56884ab53c55e51987fe93d32e33dce5e4ee654756528fd45e6ee"}
Jul 09 18:18:16.802661 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:16.802625    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8" path="/var/lib/kubelet/pods/60abb6cc-ab5b-4508-ba3f-d8fc17cd0cd8/volumes"
Jul 09 18:18:19.828630 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:19.828596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" event={"ID":"818db810-07e6-457a-8bef-ca361d746c2a","Type":"ContainerStarted","Data":"5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5"}
Jul 09 18:18:20.834023 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:20.833984    2581 generic.go:356] "Generic (PLEG): container finished" podID="818db810-07e6-457a-8bef-ca361d746c2a" containerID="5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5" exitCode=0
Jul 09 18:18:20.834513 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:20.834075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" event={"ID":"818db810-07e6-457a-8bef-ca361d746c2a","Type":"ContainerDied","Data":"5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5"}
Jul 09 18:18:21.305109 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:21.305071    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:22.845413 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:22.845375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" event={"ID":"818db810-07e6-457a-8bef-ca361d746c2a","Type":"ContainerStarted","Data":"26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"}
Jul 09 18:18:22.845758 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:22.845460    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:18:22.868791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:22.868744    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" podStartSLOduration=2.0644553979999998 podStartE2EDuration="7.868724413s" totalImagesPullingTime="5.061237489s" totalInitContainerRuntime="743.031526ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:18:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:18:22.867508883 +0000 UTC m=+896.791053355" watchObservedRunningTime="2026-07-09 18:18:22.868724413 +0000 UTC m=+896.792268888"
Jul 09 18:18:26.561998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:26.561959    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:18:26.564437 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:26.564418    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:18:42.267390 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:42.267357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:42.333849 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:42.333819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:18:53.851781 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:18:53.851754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:19:18.113039 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:18.113002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:19.394057 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:19.394027    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:31.294569 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:31.294505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:33.663537 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:33.663505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:50.542027 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:50.541942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:50.668577 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:50.668541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:51.543254 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:51.543224    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:19:51.654005 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:19:51.653964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:34.967308 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:34.967263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:35.441767 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:35.441736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:52.983416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:52.983383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:59.692887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:59.692858    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"]
Jul 09 18:20:59.703047 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:20:59.703000    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:20:59.703047 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:20:59.703047 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:20:59.703047 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447" cmd=["/bin/sleep","15"]
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:59.703078    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" message="" err=<
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:20:59.703095    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="main" containerID="cri-o://26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"
Jul 09 18:20:59.703313 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:59.703160    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="main" containerID="cri-o://26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447" gracePeriod=60
Jul 09 18:20:59.724605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:59.724566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:59.794364 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:59.794329    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:20:59.962504 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:20:59.962477    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:21:00.039383 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.039349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818db810-07e6-457a-8bef-ca361d746c2a-tls-certs\") pod \"818db810-07e6-457a-8bef-ca361d746c2a\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") "
Jul 09 18:21:00.039383 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.039395    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-tokenizer-uds\") pod \"818db810-07e6-457a-8bef-ca361d746c2a\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") "
Jul 09 18:21:00.039626 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.039429    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-kserve-provision-location\") pod \"818db810-07e6-457a-8bef-ca361d746c2a\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") "
Jul 09 18:21:00.039626 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.039461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhdwr\" (UniqueName: \"kubernetes.io/projected/818db810-07e6-457a-8bef-ca361d746c2a-kube-api-access-jhdwr\") pod \"818db810-07e6-457a-8bef-ca361d746c2a\" (UID: \"818db810-07e6-457a-8bef-ca361d746c2a\") "
Jul 09 18:21:00.039743 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.039689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "818db810-07e6-457a-8bef-ca361d746c2a" (UID: "818db810-07e6-457a-8bef-ca361d746c2a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:21:00.040094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.040068    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "818db810-07e6-457a-8bef-ca361d746c2a" (UID: "818db810-07e6-457a-8bef-ca361d746c2a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:21:00.041449 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.041427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818db810-07e6-457a-8bef-ca361d746c2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "818db810-07e6-457a-8bef-ca361d746c2a" (UID: "818db810-07e6-457a-8bef-ca361d746c2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:21:00.041650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.041629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818db810-07e6-457a-8bef-ca361d746c2a-kube-api-access-jhdwr" (OuterVolumeSpecName: "kube-api-access-jhdwr") pod "818db810-07e6-457a-8bef-ca361d746c2a" (UID: "818db810-07e6-457a-8bef-ca361d746c2a"). InnerVolumeSpecName "kube-api-access-jhdwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:21:00.140080 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.140036    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhdwr\" (UniqueName: \"kubernetes.io/projected/818db810-07e6-457a-8bef-ca361d746c2a-kube-api-access-jhdwr\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:21:00.140080 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.140072    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/818db810-07e6-457a-8bef-ca361d746c2a-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:21:00.140080 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.140087    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:21:00.140353 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.140100    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/818db810-07e6-457a-8bef-ca361d746c2a-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:21:00.610804 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.610765    2581 generic.go:356] "Generic (PLEG): container finished" podID="818db810-07e6-457a-8bef-ca361d746c2a" containerID="26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447" exitCode=0
Jul 09 18:21:00.610974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.610816    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"
Jul 09 18:21:00.610974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.610843    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" event={"ID":"818db810-07e6-457a-8bef-ca361d746c2a","Type":"ContainerDied","Data":"26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"}
Jul 09 18:21:00.610974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.610898    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv" event={"ID":"818db810-07e6-457a-8bef-ca361d746c2a","Type":"ContainerDied","Data":"57eb0db268e56884ab53c55e51987fe93d32e33dce5e4ee654756528fd45e6ee"}
Jul 09 18:21:00.610974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.610919    2581 scope.go:117] "RemoveContainer" containerID="26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"
Jul 09 18:21:00.620954 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.620933    2581 scope.go:117] "RemoveContainer" containerID="5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5"
Jul 09 18:21:00.629272 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.629251    2581 scope.go:117] "RemoveContainer" containerID="26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"
Jul 09 18:21:00.629532 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:21:00.629513    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447\": container with ID starting with 26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447 not found: ID does not exist" containerID="26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"
Jul 09 18:21:00.629595 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.629543    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447"} err="failed to get container status \"26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447\": rpc error: code = NotFound desc = could not find container \"26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447\": container with ID starting with 26be2ca30688a0dc5b242af091c66b3ec0183cd29a1e315554945da908f5a447 not found: ID does not exist"
Jul 09 18:21:00.629595 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.629561    2581 scope.go:117] "RemoveContainer" containerID="5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5"
Jul 09 18:21:00.629814 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:21:00.629799    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5\": container with ID starting with 5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5 not found: ID does not exist" containerID="5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5"
Jul 09 18:21:00.629866 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.629817    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5"} err="failed to get container status \"5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5\": rpc error: code = NotFound desc = could not find container \"5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5\": container with ID starting with 5aff853fa7fb27f062916242f9156ac2023bd22f83fd91d15feb5fc1e5605df5 not found: ID does not exist"
Jul 09 18:21:00.633261 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.633241    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"]
Jul 09 18:21:00.636990 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.636969    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-57d5bcd549-64cgv"]
Jul 09 18:21:00.799710 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:00.799679    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818db810-07e6-457a-8bef-ca361d746c2a" path="/var/lib/kubelet/pods/818db810-07e6-457a-8bef-ca361d746c2a/volumes"
Jul 09 18:21:23.985172 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.985088    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"]
Jul 09 18:21:23.985787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.985684    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="main"
Jul 09 18:21:23.985787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.985705    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="main"
Jul 09 18:21:23.985787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.985740    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="storage-initializer"
Jul 09 18:21:23.985787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.985748    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="storage-initializer"
Jul 09 18:21:23.986007 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.985842    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="818db810-07e6-457a-8bef-ca361d746c2a" containerName="main"
Jul 09 18:21:23.988786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.988768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:23.991540 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.991518    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:21:23.992492 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.992472    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:21:23.992598 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.992572    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Jul 09 18:21:23.992598 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.992585    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:21:23.998838 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:23.998815    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"]
Jul 09 18:21:24.028658 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.028627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:24.151440 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.151409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.151440 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.151443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.151641 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.151481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.151641 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.151585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmrmq\" (UniqueName: \"kubernetes.io/projected/e70c889d-255b-456b-a948-4dd728b59957-kube-api-access-gmrmq\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.151641 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.151629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e70c889d-255b-456b-a948-4dd728b59957-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.151739 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.151655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmrmq\" (UniqueName: \"kubernetes.io/projected/e70c889d-255b-456b-a948-4dd728b59957-kube-api-access-gmrmq\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e70c889d-255b-456b-a948-4dd728b59957-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252602 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252602 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252602 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252546    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252728 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.252896 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.253033 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.252959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.253093 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.253068    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.254750 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.254725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.254909 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.254888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e70c889d-255b-456b-a948-4dd728b59957-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.262559 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.262541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmrmq\" (UniqueName: \"kubernetes.io/projected/e70c889d-255b-456b-a948-4dd728b59957-kube-api-access-gmrmq\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.289249 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.289221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:24.300830 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.300811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:24.441162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.441131    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"]
Jul 09 18:21:24.444596 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:21:24.444559    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70c889d_255b_456b_a948_4dd728b59957.slice/crio-16c4f7fd26800e8af661b51ad3772e13684ee20040e712800e6900b1c8dc13c2 WatchSource:0}: Error finding container 16c4f7fd26800e8af661b51ad3772e13684ee20040e712800e6900b1c8dc13c2: Status 404 returned error can't find the container with id 16c4f7fd26800e8af661b51ad3772e13684ee20040e712800e6900b1c8dc13c2
Jul 09 18:21:24.729763 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.729722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" event={"ID":"e70c889d-255b-456b-a948-4dd728b59957","Type":"ContainerStarted","Data":"2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed"}
Jul 09 18:21:24.729944 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.729794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" event={"ID":"e70c889d-255b-456b-a948-4dd728b59957","Type":"ContainerStarted","Data":"16c4f7fd26800e8af661b51ad3772e13684ee20040e712800e6900b1c8dc13c2"}
Jul 09 18:21:24.747834 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:24.747758    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" podStartSLOduration=1.747738693 podStartE2EDuration="1.747738693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:21:24.747571721 +0000 UTC m=+1078.671116196" watchObservedRunningTime="2026-07-09 18:21:24.747738693 +0000 UTC m=+1078.671283166"
Jul 09 18:21:25.301139 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:25.301107    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:34.301505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:34.301467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:34.313502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:34.313474    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:34.787583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:34.787556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:21:35.708125 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.708094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:35.907445 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.907414    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"]
Jul 09 18:21:35.912287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.912263    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:35.914777 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.914752    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:21:35.914884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.914779    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Jul 09 18:21:35.915720 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.915704    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-invalid-token-test-epp-sa-dockercfg-85fnw\""
Jul 09 18:21:35.922888 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.922868    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"]
Jul 09 18:21:35.934735 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:35.934699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:36.061547 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.061456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tokenizer-uds\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.061706 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.061587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kserve-provision-location\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.061706 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.061643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.061706 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.061676    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9tts\" (UniqueName: \"kubernetes.io/projected/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kube-api-access-r9tts\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.162677 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.162647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kserve-provision-location\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.162833 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.162689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.162833 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.162713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9tts\" (UniqueName: \"kubernetes.io/projected/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kube-api-access-r9tts\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.162833 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.162764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tokenizer-uds\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.163021 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.163003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kserve-provision-location\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.163118 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.163098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tokenizer-uds\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.165090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.165073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tls-certs\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.178716 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.178693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9tts\" (UniqueName: \"kubernetes.io/projected/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kube-api-access-r9tts\") pod \"auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") " pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.222333 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.222304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:36.360426 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.360397    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"]
Jul 09 18:21:36.362643 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:21:36.362615    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f24ea4_be58_4b1c_98ae_9b496551b7a9.slice/crio-ea2359dc5116062e1800c86d392e6ab439bd6e30211df23ca56f0792c86d4d21 WatchSource:0}: Error finding container ea2359dc5116062e1800c86d392e6ab439bd6e30211df23ca56f0792c86d4d21: Status 404 returned error can't find the container with id ea2359dc5116062e1800c86d392e6ab439bd6e30211df23ca56f0792c86d4d21
Jul 09 18:21:36.787333 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.787241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" event={"ID":"10f24ea4-be58-4b1c-98ae-9b496551b7a9","Type":"ContainerStarted","Data":"70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef"}
Jul 09 18:21:36.787333 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:36.787294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" event={"ID":"10f24ea4-be58-4b1c-98ae-9b496551b7a9","Type":"ContainerStarted","Data":"ea2359dc5116062e1800c86d392e6ab439bd6e30211df23ca56f0792c86d4d21"}
Jul 09 18:21:37.794364 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:37.794333    2581 generic.go:356] "Generic (PLEG): container finished" podID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerID="70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef" exitCode=0
Jul 09 18:21:37.794749 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:37.794394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" event={"ID":"10f24ea4-be58-4b1c-98ae-9b496551b7a9","Type":"ContainerDied","Data":"70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef"}
Jul 09 18:21:37.889381 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:37.889347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:21:38.800765 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:38.800732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" event={"ID":"10f24ea4-be58-4b1c-98ae-9b496551b7a9","Type":"ContainerStarted","Data":"cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"}
Jul 09 18:21:38.801140 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:38.800843    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:21:38.822645 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:38.822589    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" podStartSLOduration=3.185854796 podStartE2EDuration="3.822571219s" totalImagesPullingTime="0s" totalInitContainerRuntime="636.716423ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:21:38.820493631 +0000 UTC m=+1092.744038102" watchObservedRunningTime="2026-07-09 18:21:38.822571219 +0000 UTC m=+1092.746115692"
Jul 09 18:21:59.284782 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:21:59.284743    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:09.813888 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:09.813857    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:22:14.274640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:14.274608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:20.499525 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:20.499483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:22.609563 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:22.609533    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:24.731878 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:24.731841    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"]
Jul 09 18:22:24.744067 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:24.744039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:24.833722 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:24.833694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:39.752692 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:39.752637    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" podUID="e70c889d-255b-456b-a948-4dd728b59957" containerName="main" containerID="cri-o://2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed" gracePeriod=45
Jul 09 18:22:39.777377 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:39.777338    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" podUID="e70c889d-255b-456b-a948-4dd728b59957" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/health\": dial tcp 10.134.0.33:8000: connect: connection refused"
Jul 09 18:22:39.993212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:39.993174    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:22:40.045348 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045271    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e70c889d-255b-456b-a948-4dd728b59957-tls-certs\") pod \"e70c889d-255b-456b-a948-4dd728b59957\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") "
Jul 09 18:22:40.045348 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045336    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-model-cache\") pod \"e70c889d-255b-456b-a948-4dd728b59957\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") "
Jul 09 18:22:40.045567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045356    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-dshm\") pod \"e70c889d-255b-456b-a948-4dd728b59957\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") "
Jul 09 18:22:40.045567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045385    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-tmp-dir\") pod \"e70c889d-255b-456b-a948-4dd728b59957\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") "
Jul 09 18:22:40.045567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045404    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-home\") pod \"e70c889d-255b-456b-a948-4dd728b59957\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") "
Jul 09 18:22:40.045567 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmrmq\" (UniqueName: \"kubernetes.io/projected/e70c889d-255b-456b-a948-4dd728b59957-kube-api-access-gmrmq\") pod \"e70c889d-255b-456b-a948-4dd728b59957\" (UID: \"e70c889d-255b-456b-a948-4dd728b59957\") "
Jul 09 18:22:40.045753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-model-cache" (OuterVolumeSpecName: "model-cache") pod "e70c889d-255b-456b-a948-4dd728b59957" (UID: "e70c889d-255b-456b-a948-4dd728b59957"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:22:40.045753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045659    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e70c889d-255b-456b-a948-4dd728b59957" (UID: "e70c889d-255b-456b-a948-4dd728b59957"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:22:40.045839 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045746    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-home" (OuterVolumeSpecName: "home") pod "e70c889d-255b-456b-a948-4dd728b59957" (UID: "e70c889d-255b-456b-a948-4dd728b59957"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:22:40.045839 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045821    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:22:40.045839 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045839    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:22:40.045990 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.045851    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:22:40.047495 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.047469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-dshm" (OuterVolumeSpecName: "dshm") pod "e70c889d-255b-456b-a948-4dd728b59957" (UID: "e70c889d-255b-456b-a948-4dd728b59957"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:22:40.047599 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.047525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e70c889d-255b-456b-a948-4dd728b59957-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e70c889d-255b-456b-a948-4dd728b59957" (UID: "e70c889d-255b-456b-a948-4dd728b59957"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:22:40.047599 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.047578    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70c889d-255b-456b-a948-4dd728b59957-kube-api-access-gmrmq" (OuterVolumeSpecName: "kube-api-access-gmrmq") pod "e70c889d-255b-456b-a948-4dd728b59957" (UID: "e70c889d-255b-456b-a948-4dd728b59957"). InnerVolumeSpecName "kube-api-access-gmrmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:22:40.113079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.113044    2581 generic.go:356] "Generic (PLEG): container finished" podID="e70c889d-255b-456b-a948-4dd728b59957" containerID="2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed" exitCode=0
Jul 09 18:22:40.113233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.113082    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"
Jul 09 18:22:40.113233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.113126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" event={"ID":"e70c889d-255b-456b-a948-4dd728b59957","Type":"ContainerDied","Data":"2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed"}
Jul 09 18:22:40.113233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.113173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr" event={"ID":"e70c889d-255b-456b-a948-4dd728b59957","Type":"ContainerDied","Data":"16c4f7fd26800e8af661b51ad3772e13684ee20040e712800e6900b1c8dc13c2"}
Jul 09 18:22:40.113233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.113209    2581 scope.go:117] "RemoveContainer" containerID="2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed"
Jul 09 18:22:40.122346 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.122323    2581 scope.go:117] "RemoveContainer" containerID="2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed"
Jul 09 18:22:40.122605 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:22:40.122585    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed\": container with ID starting with 2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed not found: ID does not exist" containerID="2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed"
Jul 09 18:22:40.122675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.122617    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed"} err="failed to get container status \"2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed\": rpc error: code = NotFound desc = could not find container \"2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed\": container with ID starting with 2d11bbdb7fca664788c7ded5772c697dadeb2ec0492a17b4b69ed2931a3ef7ed not found: ID does not exist"
Jul 09 18:22:40.135361 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.135335    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"]
Jul 09 18:22:40.138480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.138457    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddhsjkr"]
Jul 09 18:22:40.146947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.146930    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e70c889d-255b-456b-a948-4dd728b59957-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:22:40.147015 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.146951    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmrmq\" (UniqueName: \"kubernetes.io/projected/e70c889d-255b-456b-a948-4dd728b59957-kube-api-access-gmrmq\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:22:40.147015 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.146961    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e70c889d-255b-456b-a948-4dd728b59957-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:22:40.799779 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:40.799748    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70c889d-255b-456b-a948-4dd728b59957" path="/var/lib/kubelet/pods/e70c889d-255b-456b-a948-4dd728b59957/volumes"
Jul 09 18:22:42.332842 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:42.332807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:22:42.396733 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:22:42.396696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:21.928456 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:21.928424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:22.232689 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:22.232606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:26.597735 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:26.597710    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:23:26.598173 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:26.598010    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:23:31.887454 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:31.887423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:37.875076 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:37.875039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:37.893086 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:37.893056    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"]
Jul 09 18:23:37.902440 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:23:37.902401    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:23:37.902440 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:23:37.902440 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:23:37.902440 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a" cmd=["/bin/sleep","15"]
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:37.902458    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" message="" err=<
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:23:37.902471    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="main" containerID="cri-o://cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"
Jul 09 18:23:37.902683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:37.902506    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="main" containerID="cri-o://cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a" gracePeriod=60
Jul 09 18:23:37.966574 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:37.966544    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:38.154460 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.154436    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:23:38.253826 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.253793    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tls-certs\") pod \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") "
Jul 09 18:23:38.253826 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.253832    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9tts\" (UniqueName: \"kubernetes.io/projected/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kube-api-access-r9tts\") pod \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") "
Jul 09 18:23:38.254048 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.253898    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tokenizer-uds\") pod \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") "
Jul 09 18:23:38.254048 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.253926    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kserve-provision-location\") pod \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\" (UID: \"10f24ea4-be58-4b1c-98ae-9b496551b7a9\") "
Jul 09 18:23:38.254246 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.254171    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "10f24ea4-be58-4b1c-98ae-9b496551b7a9" (UID: "10f24ea4-be58-4b1c-98ae-9b496551b7a9"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:23:38.254684 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.254662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "10f24ea4-be58-4b1c-98ae-9b496551b7a9" (UID: "10f24ea4-be58-4b1c-98ae-9b496551b7a9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:23:38.256018 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.255996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kube-api-access-r9tts" (OuterVolumeSpecName: "kube-api-access-r9tts") pod "10f24ea4-be58-4b1c-98ae-9b496551b7a9" (UID: "10f24ea4-be58-4b1c-98ae-9b496551b7a9"). InnerVolumeSpecName "kube-api-access-r9tts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:23:38.256089 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.256018    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "10f24ea4-be58-4b1c-98ae-9b496551b7a9" (UID: "10f24ea4-be58-4b1c-98ae-9b496551b7a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:23:38.354897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.354862    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:23:38.354897 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.354894    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9tts\" (UniqueName: \"kubernetes.io/projected/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kube-api-access-r9tts\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:23:38.355099 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.354909    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:23:38.355099 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.354922    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10f24ea4-be58-4b1c-98ae-9b496551b7a9-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:23:38.394514 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.394435    2581 generic.go:356] "Generic (PLEG): container finished" podID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerID="cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a" exitCode=0
Jul 09 18:23:38.394514 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.394493    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"
Jul 09 18:23:38.394679 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.394510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" event={"ID":"10f24ea4-be58-4b1c-98ae-9b496551b7a9","Type":"ContainerDied","Data":"cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"}
Jul 09 18:23:38.394679 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.394570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s" event={"ID":"10f24ea4-be58-4b1c-98ae-9b496551b7a9","Type":"ContainerDied","Data":"ea2359dc5116062e1800c86d392e6ab439bd6e30211df23ca56f0792c86d4d21"}
Jul 09 18:23:38.394679 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.394597    2581 scope.go:117] "RemoveContainer" containerID="cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"
Jul 09 18:23:38.404109 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.404090    2581 scope.go:117] "RemoveContainer" containerID="70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef"
Jul 09 18:23:38.411913 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.411897    2581 scope.go:117] "RemoveContainer" containerID="cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"
Jul 09 18:23:38.412160 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:23:38.412142    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a\": container with ID starting with cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a not found: ID does not exist" containerID="cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"
Jul 09 18:23:38.412229 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.412172    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a"} err="failed to get container status \"cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a\": rpc error: code = NotFound desc = could not find container \"cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a\": container with ID starting with cf819fbf78d1432660fc7cadbae57af7ce956b368f8244dd0afe2aefbdc5e09a not found: ID does not exist"
Jul 09 18:23:38.412229 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.412204    2581 scope.go:117] "RemoveContainer" containerID="70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef"
Jul 09 18:23:38.412476 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:23:38.412459    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef\": container with ID starting with 70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef not found: ID does not exist" containerID="70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef"
Jul 09 18:23:38.412524 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.412481    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef"} err="failed to get container status \"70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef\": rpc error: code = NotFound desc = could not find container \"70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef\": container with ID starting with 70f8db9fe594556e21362bd68423a779c0e9c6253c7acb8f577262c8e19206ef not found: ID does not exist"
Jul 09 18:23:38.416422 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.416399    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"]
Jul 09 18:23:38.419926 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.419902    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-844f5899-v995s"]
Jul 09 18:23:38.800308 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:38.800227    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" path="/var/lib/kubelet/pods/10f24ea4-be58-4b1c-98ae-9b496551b7a9/volumes"
Jul 09 18:23:56.800799 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.800770    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"]
Jul 09 18:23:56.801181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801109    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70c889d-255b-456b-a948-4dd728b59957" containerName="main"
Jul 09 18:23:56.801181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801120    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70c889d-255b-456b-a948-4dd728b59957" containerName="main"
Jul 09 18:23:56.801181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801128    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="main"
Jul 09 18:23:56.801181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801133    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="main"
Jul 09 18:23:56.801181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801144    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="storage-initializer"
Jul 09 18:23:56.801181 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801150    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="storage-initializer"
Jul 09 18:23:56.801400 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801241    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70c889d-255b-456b-a948-4dd728b59957" containerName="main"
Jul 09 18:23:56.801400 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.801249    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="10f24ea4-be58-4b1c-98ae-9b496551b7a9" containerName="main"
Jul 09 18:23:56.804069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.804051    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.807794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.807771    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Jul 09 18:23:56.807794 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.807785    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:23:56.807989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.807807    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:23:56.807989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.807809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:23:56.815301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.815280    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"]
Jul 09 18:23:56.823196 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.823150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.823334 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.823212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de3cabd-0049-438c-8668-406c59d4c99f-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.823334 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.823238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xv26\" (UniqueName: \"kubernetes.io/projected/9de3cabd-0049-438c-8668-406c59d4c99f-kube-api-access-7xv26\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.823334 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.823311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.823451 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.823368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.823451 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.823436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.825175 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.825146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:56.924022 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.923989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924223 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de3cabd-0049-438c-8668-406c59d4c99f-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924223 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xv26\" (UniqueName: \"kubernetes.io/projected/9de3cabd-0049-438c-8668-406c59d4c99f-kube-api-access-7xv26\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924223 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924577 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924708 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.924770 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.924750    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.926620 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.926599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.926925 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.926903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de3cabd-0049-438c-8668-406c59d4c99f-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:56.933748 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:56.933725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xv26\" (UniqueName: \"kubernetes.io/projected/9de3cabd-0049-438c-8668-406c59d4c99f-kube-api-access-7xv26\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:57.084624 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:57.084541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:23:57.114614 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:57.114579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:23:57.249846 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:57.249822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"]
Jul 09 18:23:57.251920 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:23:57.251893    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de3cabd_0049_438c_8668_406c59d4c99f.slice/crio-cbad73c9a68ad735681652c26a1f8c72a761f76e6f4ef56175783d501dba0103 WatchSource:0}: Error finding container cbad73c9a68ad735681652c26a1f8c72a761f76e6f4ef56175783d501dba0103: Status 404 returned error can't find the container with id cbad73c9a68ad735681652c26a1f8c72a761f76e6f4ef56175783d501dba0103
Jul 09 18:23:57.494304 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:57.494273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh" event={"ID":"9de3cabd-0049-438c-8668-406c59d4c99f","Type":"ContainerStarted","Data":"ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9"}
Jul 09 18:23:57.494508 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:57.494322    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh" event={"ID":"9de3cabd-0049-438c-8668-406c59d4c99f","Type":"ContainerStarted","Data":"cbad73c9a68ad735681652c26a1f8c72a761f76e6f4ef56175783d501dba0103"}
Jul 09 18:23:57.514778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:57.514728    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh" podStartSLOduration=1.514713431 podStartE2EDuration="1.514713431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:23:57.513340397 +0000 UTC m=+1231.436884862" watchObservedRunningTime="2026-07-09 18:23:57.514713431 +0000 UTC m=+1231.438257950"
Jul 09 18:23:58.114732 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:23:58.114696    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:24:07.115364 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:07.115330    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:24:07.127818 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:07.127796    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:24:07.552363 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:07.552284    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:24:07.658750 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:07.658716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:17.911109 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:17.911077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:18.206519 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.206422    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"]
Jul 09 18:24:18.209978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.209959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.212881 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.212856    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-disabled-test-epp-sa-dockercfg-v6mk7\""
Jul 09 18:24:18.213775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.213756    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:24:18.213885 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.213758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"auth-disabled-test-kserve-self-signed-certs\""
Jul 09 18:24:18.222043 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.222012    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"]
Jul 09 18:24:18.247050 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.247025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:18.310264 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.310235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-tokenizer-uds\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.310417 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.310270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-kserve-provision-location\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.310417 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.310311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhzvp\" (UniqueName: \"kubernetes.io/projected/335f22cd-99e8-47fa-a411-d63593c3e57d-kube-api-access-nhzvp\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.310417 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.310340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335f22cd-99e8-47fa-a411-d63593c3e57d-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.411609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.411567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-tokenizer-uds\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.411609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.411612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-kserve-provision-location\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.411845 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.411661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhzvp\" (UniqueName: \"kubernetes.io/projected/335f22cd-99e8-47fa-a411-d63593c3e57d-kube-api-access-nhzvp\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.411845 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.411689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335f22cd-99e8-47fa-a411-d63593c3e57d-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.412033 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.412007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-tokenizer-uds\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.412071 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.412015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-kserve-provision-location\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.414159 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.414142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335f22cd-99e8-47fa-a411-d63593c3e57d-tls-certs\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.423317 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.423293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhzvp\" (UniqueName: \"kubernetes.io/projected/335f22cd-99e8-47fa-a411-d63593c3e57d-kube-api-access-nhzvp\") pod \"auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") " pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.520047 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.519963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:18.659136 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:18.659112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"]
Jul 09 18:24:18.661473 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:24:18.661445    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod335f22cd_99e8_47fa_a411_d63593c3e57d.slice/crio-46813e3bde7b583ef693d9fd015fb24c4c90ef69e0d94750b7c82fb86f3d918d WatchSource:0}: Error finding container 46813e3bde7b583ef693d9fd015fb24c4c90ef69e0d94750b7c82fb86f3d918d: Status 404 returned error can't find the container with id 46813e3bde7b583ef693d9fd015fb24c4c90ef69e0d94750b7c82fb86f3d918d
Jul 09 18:24:19.600522 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:19.600487    2581 generic.go:356] "Generic (PLEG): container finished" podID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerID="11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c" exitCode=0
Jul 09 18:24:19.600906 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:19.600578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" event={"ID":"335f22cd-99e8-47fa-a411-d63593c3e57d","Type":"ContainerDied","Data":"11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c"}
Jul 09 18:24:19.600906 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:19.600626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" event={"ID":"335f22cd-99e8-47fa-a411-d63593c3e57d","Type":"ContainerStarted","Data":"46813e3bde7b583ef693d9fd015fb24c4c90ef69e0d94750b7c82fb86f3d918d"}
Jul 09 18:24:20.607032 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:20.606999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" event={"ID":"335f22cd-99e8-47fa-a411-d63593c3e57d","Type":"ContainerStarted","Data":"78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"}
Jul 09 18:24:20.607424 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:20.607153    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:20.632069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:20.632010    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" podStartSLOduration=2.000661836 podStartE2EDuration="2.631990774s" totalImagesPullingTime="0s" totalInitContainerRuntime="631.328938ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:24:20.629585203 +0000 UTC m=+1254.553129667" watchObservedRunningTime="2026-07-09 18:24:20.631990774 +0000 UTC m=+1254.555535249"
Jul 09 18:24:28.190027 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:28.189985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:32.228233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:32.228175    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"]
Jul 09 18:24:32.228719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:32.228399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:32.319456 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:32.319412    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:47.250708 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.250661    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh" podUID="9de3cabd-0049-438c-8668-406c59d4c99f" containerName="main" containerID="cri-o://ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9" gracePeriod=45
Jul 09 18:24:47.497616 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.497590    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:24:47.583526 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583441    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de3cabd-0049-438c-8668-406c59d4c99f-tls-certs\") pod \"9de3cabd-0049-438c-8668-406c59d4c99f\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") "
Jul 09 18:24:47.583526 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583473    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xv26\" (UniqueName: \"kubernetes.io/projected/9de3cabd-0049-438c-8668-406c59d4c99f-kube-api-access-7xv26\") pod \"9de3cabd-0049-438c-8668-406c59d4c99f\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") "
Jul 09 18:24:47.583526 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583493    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-tmp-dir\") pod \"9de3cabd-0049-438c-8668-406c59d4c99f\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") "
Jul 09 18:24:47.583526 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-dshm\") pod \"9de3cabd-0049-438c-8668-406c59d4c99f\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") "
Jul 09 18:24:47.583860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583541    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-model-cache\") pod \"9de3cabd-0049-438c-8668-406c59d4c99f\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") "
Jul 09 18:24:47.583860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583627    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-home\") pod \"9de3cabd-0049-438c-8668-406c59d4c99f\" (UID: \"9de3cabd-0049-438c-8668-406c59d4c99f\") "
Jul 09 18:24:47.583860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583745    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9de3cabd-0049-438c-8668-406c59d4c99f" (UID: "9de3cabd-0049-438c-8668-406c59d4c99f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:24:47.583860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-model-cache" (OuterVolumeSpecName: "model-cache") pod "9de3cabd-0049-438c-8668-406c59d4c99f" (UID: "9de3cabd-0049-438c-8668-406c59d4c99f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:24:47.584069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583889    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-home" (OuterVolumeSpecName: "home") pod "9de3cabd-0049-438c-8668-406c59d4c99f" (UID: "9de3cabd-0049-438c-8668-406c59d4c99f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:24:47.584069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583912    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:24:47.584069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.583930    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:24:47.585605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.585577    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-dshm" (OuterVolumeSpecName: "dshm") pod "9de3cabd-0049-438c-8668-406c59d4c99f" (UID: "9de3cabd-0049-438c-8668-406c59d4c99f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:24:47.585605 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.585588    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de3cabd-0049-438c-8668-406c59d4c99f-kube-api-access-7xv26" (OuterVolumeSpecName: "kube-api-access-7xv26") pod "9de3cabd-0049-438c-8668-406c59d4c99f" (UID: "9de3cabd-0049-438c-8668-406c59d4c99f"). InnerVolumeSpecName "kube-api-access-7xv26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:24:47.585762 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.585644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de3cabd-0049-438c-8668-406c59d4c99f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9de3cabd-0049-438c-8668-406c59d4c99f" (UID: "9de3cabd-0049-438c-8668-406c59d4c99f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:24:47.684887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.684852    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:24:47.684887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.684882    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de3cabd-0049-438c-8668-406c59d4c99f-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:24:47.684887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.684893    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xv26\" (UniqueName: \"kubernetes.io/projected/9de3cabd-0049-438c-8668-406c59d4c99f-kube-api-access-7xv26\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:24:47.685105 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.684901    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de3cabd-0049-438c-8668-406c59d4c99f-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:24:47.736292 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.736258    2581 generic.go:356] "Generic (PLEG): container finished" podID="9de3cabd-0049-438c-8668-406c59d4c99f" containerID="ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9" exitCode=0
Jul 09 18:24:47.736432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.736311    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"
Jul 09 18:24:47.736432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.736355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh" event={"ID":"9de3cabd-0049-438c-8668-406c59d4c99f","Type":"ContainerDied","Data":"ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9"}
Jul 09 18:24:47.736432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.736406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh" event={"ID":"9de3cabd-0049-438c-8668-406c59d4c99f","Type":"ContainerDied","Data":"cbad73c9a68ad735681652c26a1f8c72a761f76e6f4ef56175783d501dba0103"}
Jul 09 18:24:47.736432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.736430    2581 scope.go:117] "RemoveContainer" containerID="ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9"
Jul 09 18:24:47.745645 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.745629    2581 scope.go:117] "RemoveContainer" containerID="ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9"
Jul 09 18:24:47.745900 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:24:47.745882    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9\": container with ID starting with ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9 not found: ID does not exist" containerID="ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9"
Jul 09 18:24:47.745939 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.745907    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9"} err="failed to get container status \"ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9\": rpc error: code = NotFound desc = could not find container \"ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9\": container with ID starting with ea1e42d202f139eee8be82624d8cbe90014b8178d6e2e7c970d99e48afd4a9b9 not found: ID does not exist"
Jul 09 18:24:47.758415 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.758394    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"]
Jul 09 18:24:47.764741 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:47.764721    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f4zgjh"]
Jul 09 18:24:48.799578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:48.799543    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9de3cabd-0049-438c-8668-406c59d4c99f" path="/var/lib/kubelet/pods/9de3cabd-0049-438c-8668-406c59d4c99f/volumes"
Jul 09 18:24:51.613997 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:51.613967    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:24:52.394022 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:52.393990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:24:52.497114 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:24:52.497079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:25:23.825752 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:25:23.825720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:25:24.160817 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:25:24.160768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:08.223325 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:08.223291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:23.372874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.372842    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"]
Jul 09 18:26:23.380873 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.380841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:23.382924 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:26:23.382888    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:26:23.382924 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:26:23.382924 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:26:23.382924 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9" cmd=["/bin/sleep","15"]
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.382940    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" message="" err=<
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:26:23.382954    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="main" containerID="cri-o://78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"
Jul 09 18:26:23.383228 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.383009    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="main" containerID="cri-o://78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9" gracePeriod=60
Jul 09 18:26:23.456295 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.456262    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:23.636744 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.636679    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:26:23.742787 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.742754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335f22cd-99e8-47fa-a411-d63593c3e57d-tls-certs\") pod \"335f22cd-99e8-47fa-a411-d63593c3e57d\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") "
Jul 09 18:26:23.742981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.742803    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-tokenizer-uds\") pod \"335f22cd-99e8-47fa-a411-d63593c3e57d\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") "
Jul 09 18:26:23.742981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.742846    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-kserve-provision-location\") pod \"335f22cd-99e8-47fa-a411-d63593c3e57d\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") "
Jul 09 18:26:23.742981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.742866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhzvp\" (UniqueName: \"kubernetes.io/projected/335f22cd-99e8-47fa-a411-d63593c3e57d-kube-api-access-nhzvp\") pod \"335f22cd-99e8-47fa-a411-d63593c3e57d\" (UID: \"335f22cd-99e8-47fa-a411-d63593c3e57d\") "
Jul 09 18:26:23.743212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.743155    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "335f22cd-99e8-47fa-a411-d63593c3e57d" (UID: "335f22cd-99e8-47fa-a411-d63593c3e57d"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:26:23.743566 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.743538    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "335f22cd-99e8-47fa-a411-d63593c3e57d" (UID: "335f22cd-99e8-47fa-a411-d63593c3e57d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:26:23.744970 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.744941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/335f22cd-99e8-47fa-a411-d63593c3e57d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "335f22cd-99e8-47fa-a411-d63593c3e57d" (UID: "335f22cd-99e8-47fa-a411-d63593c3e57d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:26:23.745054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.744974    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335f22cd-99e8-47fa-a411-d63593c3e57d-kube-api-access-nhzvp" (OuterVolumeSpecName: "kube-api-access-nhzvp") pod "335f22cd-99e8-47fa-a411-d63593c3e57d" (UID: "335f22cd-99e8-47fa-a411-d63593c3e57d"). InnerVolumeSpecName "kube-api-access-nhzvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:26:23.844287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.844255    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/335f22cd-99e8-47fa-a411-d63593c3e57d-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:26:23.844287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.844285    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:26:23.844287 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.844298    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/335f22cd-99e8-47fa-a411-d63593c3e57d-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:26:23.844564 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:23.844311    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhzvp\" (UniqueName: \"kubernetes.io/projected/335f22cd-99e8-47fa-a411-d63593c3e57d-kube-api-access-nhzvp\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:26:24.198773 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.198732    2581 generic.go:356] "Generic (PLEG): container finished" podID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerID="78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9" exitCode=0
Jul 09 18:26:24.198950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.198790    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"
Jul 09 18:26:24.198950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.198814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" event={"ID":"335f22cd-99e8-47fa-a411-d63593c3e57d","Type":"ContainerDied","Data":"78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"}
Jul 09 18:26:24.198950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.198866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg" event={"ID":"335f22cd-99e8-47fa-a411-d63593c3e57d","Type":"ContainerDied","Data":"46813e3bde7b583ef693d9fd015fb24c4c90ef69e0d94750b7c82fb86f3d918d"}
Jul 09 18:26:24.198950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.198886    2581 scope.go:117] "RemoveContainer" containerID="78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"
Jul 09 18:26:24.208345 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.208330    2581 scope.go:117] "RemoveContainer" containerID="11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c"
Jul 09 18:26:24.216035 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.216016    2581 scope.go:117] "RemoveContainer" containerID="78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"
Jul 09 18:26:24.216326 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:26:24.216296    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9\": container with ID starting with 78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9 not found: ID does not exist" containerID="78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"
Jul 09 18:26:24.216397 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.216334    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9"} err="failed to get container status \"78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9\": rpc error: code = NotFound desc = could not find container \"78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9\": container with ID starting with 78293b1bbdc5352af4937cbaf8cbc2bb49156bbda5aed2e089d668837e5684d9 not found: ID does not exist"
Jul 09 18:26:24.216397 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.216353    2581 scope.go:117] "RemoveContainer" containerID="11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c"
Jul 09 18:26:24.216603 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:26:24.216584    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c\": container with ID starting with 11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c not found: ID does not exist" containerID="11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c"
Jul 09 18:26:24.216650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.216610    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c"} err="failed to get container status \"11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c\": rpc error: code = NotFound desc = could not find container \"11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c\": container with ID starting with 11cf4517ea7c8b70f57901fd600e2581646fcc3e5555f7d662858c8e07c43d3c not found: ID does not exist"
Jul 09 18:26:24.228632 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.228611    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"]
Jul 09 18:26:24.233391 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.233373    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-75554c9b74-kzxdg"]
Jul 09 18:26:24.800560 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:24.800525    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" path="/var/lib/kubelet/pods/335f22cd-99e8-47fa-a411-d63593c3e57d/volumes"
Jul 09 18:26:40.687161 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:40.687117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:58.600678 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.600646    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"]
Jul 09 18:26:58.601034 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601026    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de3cabd-0049-438c-8668-406c59d4c99f" containerName="main"
Jul 09 18:26:58.601079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601038    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de3cabd-0049-438c-8668-406c59d4c99f" containerName="main"
Jul 09 18:26:58.601079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601045    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="main"
Jul 09 18:26:58.601079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601050    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="main"
Jul 09 18:26:58.601079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601065    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="storage-initializer"
Jul 09 18:26:58.601079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601071    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="storage-initializer"
Jul 09 18:26:58.601257 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601130    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de3cabd-0049-438c-8668-406c59d4c99f" containerName="main"
Jul 09 18:26:58.601257 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.601141    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="335f22cd-99e8-47fa-a411-d63593c3e57d" containerName="main"
Jul 09 18:26:58.604124 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.604106    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.609061 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.609041    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:26:58.610028 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.610004    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:26:58.610129 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.610029    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:26:58.610129 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.610050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Jul 09 18:26:58.637801 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.637767    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"]
Jul 09 18:26:58.677072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.677041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:26:58.678156 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.678136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.678265 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.678250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.678324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.678274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgrv2\" (UniqueName: \"kubernetes.io/projected/81f7495e-41da-4ba4-8ac1-709cbfb403b9-kube-api-access-mgrv2\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.678324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.678296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.678408 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.678357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.678408 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.678376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779328 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779498 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779498 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgrv2\" (UniqueName: \"kubernetes.io/projected/81f7495e-41da-4ba4-8ac1-709cbfb403b9-kube-api-access-mgrv2\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779498 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779498 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779643 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779927 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.779927 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779912    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.780121 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.779981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.781697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.781669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.781945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.781929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.796471 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.796448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgrv2\" (UniqueName: \"kubernetes.io/projected/81f7495e-41da-4ba4-8ac1-709cbfb403b9-kube-api-access-mgrv2\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:58.914432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:58.914329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:26:59.049819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:59.049793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"]
Jul 09 18:26:59.051299 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:26:59.051269    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f7495e_41da_4ba4_8ac1_709cbfb403b9.slice/crio-f2021ea8b546b499d53354785f3ec196dc0e2025ab6c25f8d65ddc625342e643 WatchSource:0}: Error finding container f2021ea8b546b499d53354785f3ec196dc0e2025ab6c25f8d65ddc625342e643: Status 404 returned error can't find the container with id f2021ea8b546b499d53354785f3ec196dc0e2025ab6c25f8d65ddc625342e643
Jul 09 18:26:59.368795 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:59.368757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" event={"ID":"81f7495e-41da-4ba4-8ac1-709cbfb403b9","Type":"ContainerStarted","Data":"43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13"}
Jul 09 18:26:59.368965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:59.368812    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" event={"ID":"81f7495e-41da-4ba4-8ac1-709cbfb403b9","Type":"ContainerStarted","Data":"f2021ea8b546b499d53354785f3ec196dc0e2025ab6c25f8d65ddc625342e643"}
Jul 09 18:26:59.392676 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:59.392622    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" podStartSLOduration=1.392606725 podStartE2EDuration="1.392606725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:26:59.390107313 +0000 UTC m=+1413.313651762" watchObservedRunningTime="2026-07-09 18:26:59.392606725 +0000 UTC m=+1413.316151253"
Jul 09 18:26:59.915300 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:26:59.915271    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:27:08.915140 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:27:08.915102    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:27:08.927812 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:27:08.927787    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:27:09.426593 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:27:09.426565    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:27:27.022939 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:27:27.022862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:27.121160 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:27:27.121121    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:27:28.780619 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:27:28.780590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:02.119072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:02.119040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:02.414702 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:02.414614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:10.108688 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:10.108650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:11.390101 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:11.390062    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"]
Jul 09 18:28:11.414989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:11.414958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:24.408293 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:24.408246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:26.410110 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.410034    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" podUID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" containerName="main" containerID="cri-o://43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13" gracePeriod=45
Jul 09 18:28:26.415989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.415956    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" podUID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Jul 09 18:28:26.630590 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.630567    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:28:26.631130 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.631093    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:28:26.667297 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.667243    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:28:26.777340 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777306    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-home\") pod \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") "
Jul 09 18:28:26.777505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777396    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgrv2\" (UniqueName: \"kubernetes.io/projected/81f7495e-41da-4ba4-8ac1-709cbfb403b9-kube-api-access-mgrv2\") pod \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") "
Jul 09 18:28:26.777505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-dshm\") pod \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") "
Jul 09 18:28:26.777505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777446    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tmp-dir\") pod \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") "
Jul 09 18:28:26.777505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777465    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-model-cache\") pod \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") "
Jul 09 18:28:26.777505 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777505    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tls-certs\") pod \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\" (UID: \"81f7495e-41da-4ba4-8ac1-709cbfb403b9\") "
Jul 09 18:28:26.777770 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777612    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-home" (OuterVolumeSpecName: "home") pod "81f7495e-41da-4ba4-8ac1-709cbfb403b9" (UID: "81f7495e-41da-4ba4-8ac1-709cbfb403b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:26.777770 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777706    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81f7495e-41da-4ba4-8ac1-709cbfb403b9" (UID: "81f7495e-41da-4ba4-8ac1-709cbfb403b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:26.777872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777764    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "81f7495e-41da-4ba4-8ac1-709cbfb403b9" (UID: "81f7495e-41da-4ba4-8ac1-709cbfb403b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:26.777926 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777882    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:28:26.777978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777924    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:28:26.777978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.777939    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:28:26.779607 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.779585    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-dshm" (OuterVolumeSpecName: "dshm") pod "81f7495e-41da-4ba4-8ac1-709cbfb403b9" (UID: "81f7495e-41da-4ba4-8ac1-709cbfb403b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:28:26.779607 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.779592    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81f7495e-41da-4ba4-8ac1-709cbfb403b9" (UID: "81f7495e-41da-4ba4-8ac1-709cbfb403b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:28:26.779725 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.779696    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f7495e-41da-4ba4-8ac1-709cbfb403b9-kube-api-access-mgrv2" (OuterVolumeSpecName: "kube-api-access-mgrv2") pod "81f7495e-41da-4ba4-8ac1-709cbfb403b9" (UID: "81f7495e-41da-4ba4-8ac1-709cbfb403b9"). InnerVolumeSpecName "kube-api-access-mgrv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:28:26.795352 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.795320    2581 generic.go:356] "Generic (PLEG): container finished" podID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" containerID="43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13" exitCode=0
Jul 09 18:28:26.796560 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.796540    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"
Jul 09 18:28:26.799894 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.799864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" event={"ID":"81f7495e-41da-4ba4-8ac1-709cbfb403b9","Type":"ContainerDied","Data":"43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13"}
Jul 09 18:28:26.800006 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.799916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5" event={"ID":"81f7495e-41da-4ba4-8ac1-709cbfb403b9","Type":"ContainerDied","Data":"f2021ea8b546b499d53354785f3ec196dc0e2025ab6c25f8d65ddc625342e643"}
Jul 09 18:28:26.800006 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.799950    2581 scope.go:117] "RemoveContainer" containerID="43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13"
Jul 09 18:28:26.808980 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.808961    2581 scope.go:117] "RemoveContainer" containerID="43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13"
Jul 09 18:28:26.809310 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:28:26.809293    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13\": container with ID starting with 43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13 not found: ID does not exist" containerID="43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13"
Jul 09 18:28:26.809354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.809319    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13"} err="failed to get container status \"43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13\": rpc error: code = NotFound desc = could not find container \"43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13\": container with ID starting with 43b226b053d91572083277c5a80bd919e6e28b4fdea26c54ed54213fa8abad13 not found: ID does not exist"
Jul 09 18:28:26.829609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.829586    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"]
Jul 09 18:28:26.844635 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.844609    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d46jww5"]
Jul 09 18:28:26.878860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.878835    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f7495e-41da-4ba4-8ac1-709cbfb403b9-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:28:26.878860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.878859    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f7495e-41da-4ba4-8ac1-709cbfb403b9-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:28:26.878974 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:26.878871    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgrv2\" (UniqueName: \"kubernetes.io/projected/81f7495e-41da-4ba4-8ac1-709cbfb403b9-kube-api-access-mgrv2\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:28:28.799880 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:28.799847    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" path="/var/lib/kubelet/pods/81f7495e-41da-4ba4-8ac1-709cbfb403b9/volumes"
Jul 09 18:28:41.038274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.038242    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:41.305909 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.305826    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"]
Jul 09 18:28:41.306324 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.306303    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" containerName="main"
Jul 09 18:28:41.306421 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.306328    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" containerName="main"
Jul 09 18:28:41.306480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.306437    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f7495e-41da-4ba4-8ac1-709cbfb403b9" containerName="main"
Jul 09 18:28:41.310939 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.310917    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.314697 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.314670    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-lw6gv\""
Jul 09 18:28:41.315631 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.315612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:28:41.315705 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.315624    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:28:41.315775 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.315757    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Jul 09 18:28:41.315879 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.315865    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:28:41.316251 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.316236    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:28:41.337963 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.337938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"]
Jul 09 18:28:41.391650 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.391621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:41.396880 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.396860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865e8923-7fe0-4041-acca-344115847a8b-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.396965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.396907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.396965 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.396934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv7fn\" (UniqueName: \"kubernetes.io/projected/865e8923-7fe0-4041-acca-344115847a8b-kube-api-access-hv7fn\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.397054 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.396975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.497502 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.497474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865e8923-7fe0-4041-acca-344115847a8b-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.497651 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.497522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.497651 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.497553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv7fn\" (UniqueName: \"kubernetes.io/projected/865e8923-7fe0-4041-acca-344115847a8b-kube-api-access-hv7fn\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.497651 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.497573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.497972 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.497947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-tokenizer-uds\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.498028 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.497962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-kserve-provision-location\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.499955 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.499927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865e8923-7fe0-4041-acca-344115847a8b-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.510973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.510949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv7fn\" (UniqueName: \"kubernetes.io/projected/865e8923-7fe0-4041-acca-344115847a8b-kube-api-access-hv7fn\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") " pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.619989 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.619956    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:41.758257 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.758233    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"]
Jul 09 18:28:41.760586 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:28:41.760558    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865e8923_7fe0_4041_acca_344115847a8b.slice/crio-90034f77ae4ca52d84dac20c162a9c456c1bc5821fdb9abcdcd9dd7f559c9e39 WatchSource:0}: Error finding container 90034f77ae4ca52d84dac20c162a9c456c1bc5821fdb9abcdcd9dd7f559c9e39: Status 404 returned error can't find the container with id 90034f77ae4ca52d84dac20c162a9c456c1bc5821fdb9abcdcd9dd7f559c9e39
Jul 09 18:28:41.865848 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.865808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" event={"ID":"865e8923-7fe0-4041-acca-344115847a8b","Type":"ContainerStarted","Data":"79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50"}
Jul 09 18:28:41.866003 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:41.865885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" event={"ID":"865e8923-7fe0-4041-acca-344115847a8b","Type":"ContainerStarted","Data":"90034f77ae4ca52d84dac20c162a9c456c1bc5821fdb9abcdcd9dd7f559c9e39"}
Jul 09 18:28:42.872847 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:42.872808    2581 generic.go:356] "Generic (PLEG): container finished" podID="865e8923-7fe0-4041-acca-344115847a8b" containerID="79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50" exitCode=0
Jul 09 18:28:42.873347 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:42.872882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" event={"ID":"865e8923-7fe0-4041-acca-344115847a8b","Type":"ContainerDied","Data":"79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50"}
Jul 09 18:28:43.880366 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:43.880330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" event={"ID":"865e8923-7fe0-4041-acca-344115847a8b","Type":"ContainerStarted","Data":"5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"}
Jul 09 18:28:43.880855 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:43.880498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:28:43.903443 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:43.903390    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" podStartSLOduration=2.261986241 podStartE2EDuration="2.903372171s" totalImagesPullingTime="0s" totalInitContainerRuntime="641.38593ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:43.900699778 +0000 UTC m=+1517.824244250" watchObservedRunningTime="2026-07-09 18:28:43.903372171 +0000 UTC m=+1517.826916645"
Jul 09 18:28:47.713791 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:47.713761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:47.816102 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:47.816073    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:28:56.504032 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:28:56.503998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:14.886679 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:14.886648    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:29:33.765386 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:33.765342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:35.024608 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:35.024573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:53.894814 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:53.894781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:54.005725 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:54.005692    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:54.070707 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:54.070678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:56.494164 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:56.494131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:56.595365 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:56.595335    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:29:57.123431 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:29:57.123401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:30:25.118283 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:30:25.118210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:30:25.409606 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:30:25.409517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:30:36.782348 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:30:36.782312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:30:36.883679 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:30:36.883642    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:30:51.936822 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:30:51.936783    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:31:08.817834 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:08.817804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:31:08.834395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:08.834367    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"]
Jul 09 18:31:08.843214 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:31:08.843162    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:31:08.843214 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:31:08.843214 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:31:08.843214 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c" cmd=["/bin/sleep","15"]
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:08.843218    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" message="" err=<
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:31:08.843232    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="main" containerID="cri-o://5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"
Jul 09 18:31:08.843458 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:08.843268    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="main" containerID="cri-o://5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c" gracePeriod=60
Jul 09 18:31:08.917633 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:08.917594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:31:09.104479 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.104456    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:31:09.233541 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.233514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-kserve-provision-location\") pod \"865e8923-7fe0-4041-acca-344115847a8b\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") "
Jul 09 18:31:09.233719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.233554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865e8923-7fe0-4041-acca-344115847a8b-tls-certs\") pod \"865e8923-7fe0-4041-acca-344115847a8b\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") "
Jul 09 18:31:09.233719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.233619    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-tokenizer-uds\") pod \"865e8923-7fe0-4041-acca-344115847a8b\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") "
Jul 09 18:31:09.233719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.233642    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv7fn\" (UniqueName: \"kubernetes.io/projected/865e8923-7fe0-4041-acca-344115847a8b-kube-api-access-hv7fn\") pod \"865e8923-7fe0-4041-acca-344115847a8b\" (UID: \"865e8923-7fe0-4041-acca-344115847a8b\") "
Jul 09 18:31:09.233895 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.233855    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "865e8923-7fe0-4041-acca-344115847a8b" (UID: "865e8923-7fe0-4041-acca-344115847a8b"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:31:09.234274 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.234242    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "865e8923-7fe0-4041-acca-344115847a8b" (UID: "865e8923-7fe0-4041-acca-344115847a8b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:31:09.235675 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.235647    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865e8923-7fe0-4041-acca-344115847a8b-kube-api-access-hv7fn" (OuterVolumeSpecName: "kube-api-access-hv7fn") pod "865e8923-7fe0-4041-acca-344115847a8b" (UID: "865e8923-7fe0-4041-acca-344115847a8b"). InnerVolumeSpecName "kube-api-access-hv7fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:31:09.235765 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.235686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865e8923-7fe0-4041-acca-344115847a8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "865e8923-7fe0-4041-acca-344115847a8b" (UID: "865e8923-7fe0-4041-acca-344115847a8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:31:09.334953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.334877    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv7fn\" (UniqueName: \"kubernetes.io/projected/865e8923-7fe0-4041-acca-344115847a8b-kube-api-access-hv7fn\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:31:09.334953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.334905    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:31:09.334953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.334916    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865e8923-7fe0-4041-acca-344115847a8b-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:31:09.334953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.334925    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/865e8923-7fe0-4041-acca-344115847a8b-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:31:09.574730 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.574696    2581 generic.go:356] "Generic (PLEG): container finished" podID="865e8923-7fe0-4041-acca-344115847a8b" containerID="5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c" exitCode=0
Jul 09 18:31:09.574917 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.574746    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"
Jul 09 18:31:09.574917 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.574790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" event={"ID":"865e8923-7fe0-4041-acca-344115847a8b","Type":"ContainerDied","Data":"5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"}
Jul 09 18:31:09.574917 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.574851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz" event={"ID":"865e8923-7fe0-4041-acca-344115847a8b","Type":"ContainerDied","Data":"90034f77ae4ca52d84dac20c162a9c456c1bc5821fdb9abcdcd9dd7f559c9e39"}
Jul 09 18:31:09.574917 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.574880    2581 scope.go:117] "RemoveContainer" containerID="5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"
Jul 09 18:31:09.584391 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.584375    2581 scope.go:117] "RemoveContainer" containerID="79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50"
Jul 09 18:31:09.592046 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.592028    2581 scope.go:117] "RemoveContainer" containerID="5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"
Jul 09 18:31:09.592339 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:31:09.592322    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c\": container with ID starting with 5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c not found: ID does not exist" containerID="5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"
Jul 09 18:31:09.592389 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.592347    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c"} err="failed to get container status \"5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c\": rpc error: code = NotFound desc = could not find container \"5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c\": container with ID starting with 5111259bf45a12138f62362af2d4ff6b3b14e20fbb2751343c004875e54a037c not found: ID does not exist"
Jul 09 18:31:09.592389 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.592361    2581 scope.go:117] "RemoveContainer" containerID="79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50"
Jul 09 18:31:09.592605 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:31:09.592587    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50\": container with ID starting with 79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50 not found: ID does not exist" containerID="79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50"
Jul 09 18:31:09.592651 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.592609    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50"} err="failed to get container status \"79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50\": rpc error: code = NotFound desc = could not find container \"79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50\": container with ID starting with 79a018a036e7c5e123386744d5cd190ff11f14af847b1060d428e3002889cd50 not found: ID does not exist"
Jul 09 18:31:09.596229 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.596210    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"]
Jul 09 18:31:09.602609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:09.602592    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-schelj9wz"]
Jul 09 18:31:10.800069 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:10.800034    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="865e8923-7fe0-4041-acca-344115847a8b" path="/var/lib/kubelet/pods/865e8923-7fe0-4041-acca-344115847a8b/volumes"
Jul 09 18:31:35.614767 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.614734    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:31:35.901372 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.901294    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"]
Jul 09 18:31:35.901672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.901659    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="main"
Jul 09 18:31:35.901714 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.901673    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="main"
Jul 09 18:31:35.901714 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.901686    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="storage-initializer"
Jul 09 18:31:35.901714 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.901692    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="storage-initializer"
Jul 09 18:31:35.901805 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.901775    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="865e8923-7fe0-4041-acca-344115847a8b" containerName="main"
Jul 09 18:31:35.904663 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.904647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:35.907131 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.907111    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:31:35.908002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.907982    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:31:35.908118 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.908089    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:31:35.908212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.908123    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:31:35.908335 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.908318    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Jul 09 18:31:35.908410 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.908319    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-9xz9s\""
Jul 09 18:31:35.915285 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.915264    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"]
Jul 09 18:31:35.927576 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.927554    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:31:35.975518 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.975487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:35.975680 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.975536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:35.975680 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.975638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd46q\" (UniqueName: \"kubernetes.io/projected/9a8efecf-e749-4682-8856-e95b8eb22017-kube-api-access-pd46q\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:35.975680 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:35.975665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a8efecf-e749-4682-8856-e95b8eb22017-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.076239 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.076182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd46q\" (UniqueName: \"kubernetes.io/projected/9a8efecf-e749-4682-8856-e95b8eb22017-kube-api-access-pd46q\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.076431 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.076246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a8efecf-e749-4682-8856-e95b8eb22017-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.076431 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.076299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.076431 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.076350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.076722 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.076702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.076785 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.076732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.078692 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.078671    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a8efecf-e749-4682-8856-e95b8eb22017-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.084962 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.084939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd46q\" (UniqueName: \"kubernetes.io/projected/9a8efecf-e749-4682-8856-e95b8eb22017-kube-api-access-pd46q\") pod \"custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.214338 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.214253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:36.353026 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.352999    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"]
Jul 09 18:31:36.354977 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:31:36.354950    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8efecf_e749_4682_8856_e95b8eb22017.slice/crio-3d948eafae89c54b25fa800e9ae8199a1a8d4bb6bf9c29f74b6cd0379b161d34 WatchSource:0}: Error finding container 3d948eafae89c54b25fa800e9ae8199a1a8d4bb6bf9c29f74b6cd0379b161d34: Status 404 returned error can't find the container with id 3d948eafae89c54b25fa800e9ae8199a1a8d4bb6bf9c29f74b6cd0379b161d34
Jul 09 18:31:36.708261 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.708216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" event={"ID":"9a8efecf-e749-4682-8856-e95b8eb22017","Type":"ContainerStarted","Data":"985fed846897f869c06ba3e49a972b66d22f1e2b1edaa90eea55fff729ff269e"}
Jul 09 18:31:36.708643 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:36.708279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" event={"ID":"9a8efecf-e749-4682-8856-e95b8eb22017","Type":"ContainerStarted","Data":"3d948eafae89c54b25fa800e9ae8199a1a8d4bb6bf9c29f74b6cd0379b161d34"}
Jul 09 18:31:37.714300 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:37.714264    2581 generic.go:356] "Generic (PLEG): container finished" podID="9a8efecf-e749-4682-8856-e95b8eb22017" containerID="985fed846897f869c06ba3e49a972b66d22f1e2b1edaa90eea55fff729ff269e" exitCode=0
Jul 09 18:31:37.714687 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:37.714340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" event={"ID":"9a8efecf-e749-4682-8856-e95b8eb22017","Type":"ContainerDied","Data":"985fed846897f869c06ba3e49a972b66d22f1e2b1edaa90eea55fff729ff269e"}
Jul 09 18:31:38.720399 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:38.720357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" event={"ID":"9a8efecf-e749-4682-8856-e95b8eb22017","Type":"ContainerStarted","Data":"1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b"}
Jul 09 18:31:38.720796 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:38.720482    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:31:38.746648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:38.746598    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" podStartSLOduration=3.06562665 podStartE2EDuration="3.746581671s" totalImagesPullingTime="0s" totalInitContainerRuntime="680.955021ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:31:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:31:38.744764772 +0000 UTC m=+1692.668309244" watchObservedRunningTime="2026-07-09 18:31:38.746581671 +0000 UTC m=+1692.670126156"
Jul 09 18:31:52.605822 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:31:52.605735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:32:09.727772 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:32:09.727742    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:32:35.063783 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:32:35.063751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:32:35.160065 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:32:35.160034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:00.081634 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:00.081606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:00.448753 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:00.448722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:15.213782 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:15.213745    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:26.661147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:26.661117    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:33:26.663747 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:26.663727    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:33:41.166850 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.166808    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"]
Jul 09 18:33:41.167872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.167845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:41.175802 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:33:41.175775    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:33:41.175802 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:33:41.175802 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:33:41.175802 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b" cmd=["/bin/sleep","15"]
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.175818    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" message="" err=<
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:33:41.175832    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="main" containerID="cri-o://1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b"
Jul 09 18:33:41.176008 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.175877    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="main" containerID="cri-o://1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b" gracePeriod=60
Jul 09 18:33:41.253609 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.253576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:33:41.311548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.311517    2581 generic.go:356] "Generic (PLEG): container finished" podID="9a8efecf-e749-4682-8856-e95b8eb22017" containerID="1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b" exitCode=0
Jul 09 18:33:41.311684 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.311562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" event={"ID":"9a8efecf-e749-4682-8856-e95b8eb22017","Type":"ContainerDied","Data":"1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b"}
Jul 09 18:33:41.434842 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.434816    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:33:41.511500 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.511469    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-tokenizer-uds\") pod \"9a8efecf-e749-4682-8856-e95b8eb22017\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") "
Jul 09 18:33:41.511683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.511514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-kserve-provision-location\") pod \"9a8efecf-e749-4682-8856-e95b8eb22017\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") "
Jul 09 18:33:41.511683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.511562    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd46q\" (UniqueName: \"kubernetes.io/projected/9a8efecf-e749-4682-8856-e95b8eb22017-kube-api-access-pd46q\") pod \"9a8efecf-e749-4682-8856-e95b8eb22017\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") "
Jul 09 18:33:41.511683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.511611    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a8efecf-e749-4682-8856-e95b8eb22017-tls-certs\") pod \"9a8efecf-e749-4682-8856-e95b8eb22017\" (UID: \"9a8efecf-e749-4682-8856-e95b8eb22017\") "
Jul 09 18:33:41.511831 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.511718    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "9a8efecf-e749-4682-8856-e95b8eb22017" (UID: "9a8efecf-e749-4682-8856-e95b8eb22017"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:33:41.511961 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.511936    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:33:41.512354 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.512328    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9a8efecf-e749-4682-8856-e95b8eb22017" (UID: "9a8efecf-e749-4682-8856-e95b8eb22017"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:33:41.513619 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.513588    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a8efecf-e749-4682-8856-e95b8eb22017-kube-api-access-pd46q" (OuterVolumeSpecName: "kube-api-access-pd46q") pod "9a8efecf-e749-4682-8856-e95b8eb22017" (UID: "9a8efecf-e749-4682-8856-e95b8eb22017"). InnerVolumeSpecName "kube-api-access-pd46q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:33:41.513713 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.513687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a8efecf-e749-4682-8856-e95b8eb22017-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a8efecf-e749-4682-8856-e95b8eb22017" (UID: "9a8efecf-e749-4682-8856-e95b8eb22017"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:33:41.612607 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.612571    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a8efecf-e749-4682-8856-e95b8eb22017-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:33:41.612607 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.612600    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9a8efecf-e749-4682-8856-e95b8eb22017-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:33:41.612607 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:41.612612    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd46q\" (UniqueName: \"kubernetes.io/projected/9a8efecf-e749-4682-8856-e95b8eb22017-kube-api-access-pd46q\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:33:42.317818 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.317791    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"
Jul 09 18:33:42.318320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.317779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv" event={"ID":"9a8efecf-e749-4682-8856-e95b8eb22017","Type":"ContainerDied","Data":"3d948eafae89c54b25fa800e9ae8199a1a8d4bb6bf9c29f74b6cd0379b161d34"}
Jul 09 18:33:42.318320 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.317926    2581 scope.go:117] "RemoveContainer" containerID="1f888bc63ed53b61fbef84070ea1c50bf4c8570c59af02b94895a92639c31a2b"
Jul 09 18:33:42.327980 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.327960    2581 scope.go:117] "RemoveContainer" containerID="985fed846897f869c06ba3e49a972b66d22f1e2b1edaa90eea55fff729ff269e"
Jul 09 18:33:42.343589 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.343563    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"]
Jul 09 18:33:42.347895 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.347873    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-c5b5f7f6rwqgv"]
Jul 09 18:33:42.806220 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:33:42.806167    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" path="/var/lib/kubelet/pods/9a8efecf-e749-4682-8856-e95b8eb22017/volumes"
Jul 09 18:34:29.784474 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:29.784444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:34:30.088081 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.087991    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"]
Jul 09 18:34:30.088413 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.088397    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="storage-initializer"
Jul 09 18:34:30.088474 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.088415    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="storage-initializer"
Jul 09 18:34:30.088474 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.088436    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="main"
Jul 09 18:34:30.088474 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.088442    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="main"
Jul 09 18:34:30.088571 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.088494    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a8efecf-e749-4682-8856-e95b8eb22017" containerName="main"
Jul 09 18:34:30.091141 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.091124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.098437 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.098415    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-epp-sa-dockercfg-dnzgg\""
Jul 09 18:34:30.099241 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.099221    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:34:30.099339 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.099290    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:34:30.099569 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.099545    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-test-kserve-self-signed-certs\""
Jul 09 18:34:30.099648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.099614    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:34:30.099648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.099633    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:34:30.110863 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.110841    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"]
Jul 09 18:34:30.147397 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.147366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.147576 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.147423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad63841-d112-4a8d-b356-14e12b8807b1-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.147576 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.147456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.147576 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.147546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjwd7\" (UniqueName: \"kubernetes.io/projected/cad63841-d112-4a8d-b356-14e12b8807b1-kube-api-access-pjwd7\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.210476 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.210444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:34:30.248524 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.248485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.248713 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.248554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjwd7\" (UniqueName: \"kubernetes.io/projected/cad63841-d112-4a8d-b356-14e12b8807b1-kube-api-access-pjwd7\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.248713 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.248596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.248713 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.248633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad63841-d112-4a8d-b356-14e12b8807b1-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.248895 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.248875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-kserve-provision-location\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.248949 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.248898    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-tokenizer-uds\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.250946 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.250927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad63841-d112-4a8d-b356-14e12b8807b1-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.264891 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.264867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjwd7\" (UniqueName: \"kubernetes.io/projected/cad63841-d112-4a8d-b356-14e12b8807b1-kube-api-access-pjwd7\") pod \"router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") " pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.386535 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.386505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:34:30.401046 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.401020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:30.539584 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.539557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"]
Jul 09 18:34:30.542012 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:34:30.541985    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad63841_d112_4a8d_b356_14e12b8807b1.slice/crio-baaec80ef8501626bc5269d85fb82eb40b5e46d2b9bbef389bafb19c21762c31 WatchSource:0}: Error finding container baaec80ef8501626bc5269d85fb82eb40b5e46d2b9bbef389bafb19c21762c31: Status 404 returned error can't find the container with id baaec80ef8501626bc5269d85fb82eb40b5e46d2b9bbef389bafb19c21762c31
Jul 09 18:34:30.557792 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:30.557770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" event={"ID":"cad63841-d112-4a8d-b356-14e12b8807b1","Type":"ContainerStarted","Data":"baaec80ef8501626bc5269d85fb82eb40b5e46d2b9bbef389bafb19c21762c31"}
Jul 09 18:34:31.570910 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:31.570823    2581 generic.go:356] "Generic (PLEG): container finished" podID="cad63841-d112-4a8d-b356-14e12b8807b1" containerID="212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0" exitCode=0
Jul 09 18:34:31.571275 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:31.570903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" event={"ID":"cad63841-d112-4a8d-b356-14e12b8807b1","Type":"ContainerDied","Data":"212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0"}
Jul 09 18:34:32.577067 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:32.577034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" event={"ID":"cad63841-d112-4a8d-b356-14e12b8807b1","Type":"ContainerStarted","Data":"15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"}
Jul 09 18:34:32.577461 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:32.577139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:34:32.598364 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:34:32.598317    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" podStartSLOduration=1.9714939139999998 podStartE2EDuration="2.598303942s" totalImagesPullingTime="0s" totalInitContainerRuntime="626.810028ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:34:32.597808524 +0000 UTC m=+1866.521353007" watchObservedRunningTime="2026-07-09 18:34:32.598303942 +0000 UTC m=+1866.521848413"
Jul 09 18:35:03.586729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:35:03.586651    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:35:05.005992 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:35:05.005965    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:35:59.630385 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:35:59.630350    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:35:59.816148 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:35:59.816114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:36:27.927532 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:36:27.927450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:36:28.128178 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:36:28.128146    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:36:28.266874 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:36:28.266797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:36:28.514340 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:36:28.514312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:36:54.996524 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:36:54.996493    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:36:58.761223 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:36:58.761175    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:08.466687 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.466653    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"]
Jul 09 18:37:08.475941 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:37:08.475902    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:37:08.475941 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:37:08.475941 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:37:08.475941 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390" cmd=["/bin/sleep","15"]
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.475954    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" message="" err=<
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:37:08.475965    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="main" containerID="cri-o://15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"
Jul 09 18:37:08.476206 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.476000    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="main" containerID="cri-o://15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390" gracePeriod=60
Jul 09 18:37:08.667736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.667696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:08.716950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.716896    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:37:08.753081 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.753053    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:08.871047 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871014    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjwd7\" (UniqueName: \"kubernetes.io/projected/cad63841-d112-4a8d-b356-14e12b8807b1-kube-api-access-pjwd7\") pod \"cad63841-d112-4a8d-b356-14e12b8807b1\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") "
Jul 09 18:37:08.871255 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad63841-d112-4a8d-b356-14e12b8807b1-tls-certs\") pod \"cad63841-d112-4a8d-b356-14e12b8807b1\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") "
Jul 09 18:37:08.871255 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871157    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-tokenizer-uds\") pod \"cad63841-d112-4a8d-b356-14e12b8807b1\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") "
Jul 09 18:37:08.871255 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871201    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-kserve-provision-location\") pod \"cad63841-d112-4a8d-b356-14e12b8807b1\" (UID: \"cad63841-d112-4a8d-b356-14e12b8807b1\") "
Jul 09 18:37:08.871445 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871404    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "cad63841-d112-4a8d-b356-14e12b8807b1" (UID: "cad63841-d112-4a8d-b356-14e12b8807b1"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:37:08.871531 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871517    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:37:08.871889 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.871864    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cad63841-d112-4a8d-b356-14e12b8807b1" (UID: "cad63841-d112-4a8d-b356-14e12b8807b1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:37:08.873106 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.873073    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad63841-d112-4a8d-b356-14e12b8807b1-kube-api-access-pjwd7" (OuterVolumeSpecName: "kube-api-access-pjwd7") pod "cad63841-d112-4a8d-b356-14e12b8807b1" (UID: "cad63841-d112-4a8d-b356-14e12b8807b1"). InnerVolumeSpecName "kube-api-access-pjwd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:37:08.873236 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.873147    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cad63841-d112-4a8d-b356-14e12b8807b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cad63841-d112-4a8d-b356-14e12b8807b1" (UID: "cad63841-d112-4a8d-b356-14e12b8807b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:37:08.971996 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.971923    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad63841-d112-4a8d-b356-14e12b8807b1-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:37:08.971996 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.971952    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad63841-d112-4a8d-b356-14e12b8807b1-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:37:08.971996 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:08.971967    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjwd7\" (UniqueName: \"kubernetes.io/projected/cad63841-d112-4a8d-b356-14e12b8807b1-kube-api-access-pjwd7\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:37:09.341837 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.341754    2581 generic.go:356] "Generic (PLEG): container finished" podID="cad63841-d112-4a8d-b356-14e12b8807b1" containerID="15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390" exitCode=0
Jul 09 18:37:09.341837 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.341804    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"
Jul 09 18:37:09.342015 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.341838    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" event={"ID":"cad63841-d112-4a8d-b356-14e12b8807b1","Type":"ContainerDied","Data":"15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"}
Jul 09 18:37:09.342015 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.341891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j" event={"ID":"cad63841-d112-4a8d-b356-14e12b8807b1","Type":"ContainerDied","Data":"baaec80ef8501626bc5269d85fb82eb40b5e46d2b9bbef389bafb19c21762c31"}
Jul 09 18:37:09.342015 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.341918    2581 scope.go:117] "RemoveContainer" containerID="15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"
Jul 09 18:37:09.351646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.351629    2581 scope.go:117] "RemoveContainer" containerID="212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0"
Jul 09 18:37:09.359282 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.359264    2581 scope.go:117] "RemoveContainer" containerID="15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"
Jul 09 18:37:09.359549 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:37:09.359530    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390\": container with ID starting with 15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390 not found: ID does not exist" containerID="15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"
Jul 09 18:37:09.359632 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.359563    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390"} err="failed to get container status \"15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390\": rpc error: code = NotFound desc = could not find container \"15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390\": container with ID starting with 15c5f3d6962d9e78b6e2767d0346c07dbe12732776411f43de4f776da2329390 not found: ID does not exist"
Jul 09 18:37:09.359632 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.359586    2581 scope.go:117] "RemoveContainer" containerID="212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0"
Jul 09 18:37:09.359816 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:37:09.359798    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0\": container with ID starting with 212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0 not found: ID does not exist" containerID="212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0"
Jul 09 18:37:09.359860 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.359824    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0"} err="failed to get container status \"212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0\": rpc error: code = NotFound desc = could not find container \"212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0\": container with ID starting with 212062df8fdfa37cb824e09b309246820d31f0bce234b03ac68f8aa8a65773e0 not found: ID does not exist"
Jul 09 18:37:09.365560 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.365538    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"]
Jul 09 18:37:09.369944 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:09.369926    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-79fb5fd958-rvg9j"]
Jul 09 18:37:10.799847 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:10.799807    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" path="/var/lib/kubelet/pods/cad63841-d112-4a8d-b356-14e12b8807b1/volumes"
Jul 09 18:37:22.103416 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:22.103382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:31.010584 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.010547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:31.190801 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.190770    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"]
Jul 09 18:37:31.191149 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.191137    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="storage-initializer"
Jul 09 18:37:31.191206 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.191151    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="storage-initializer"
Jul 09 18:37:31.191206 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.191165    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="main"
Jul 09 18:37:31.191206 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.191171    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="main"
Jul 09 18:37:31.191307 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.191246    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad63841-d112-4a8d-b356-14e12b8807b1" containerName="main"
Jul 09 18:37:31.194340 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.194322    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.196967 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.196944    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:37:31.197087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.197029    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-f9txq\""
Jul 09 18:37:31.197087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.197038    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:37:31.197225 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.197117    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:37:31.198084 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.198050    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Jul 09 18:37:31.198233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.198107    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:37:31.204975 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.204950    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"]
Jul 09 18:37:31.218119 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.218097    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:31.240632 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.240607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4kg\" (UniqueName: \"kubernetes.io/projected/e15fda24-3fce-405b-b7ee-682fcfa242e6-kube-api-access-vx4kg\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.240736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.240646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.240736 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.240668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.240839 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.240756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15fda24-3fce-405b-b7ee-682fcfa242e6-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.341210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.341111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.341210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.341156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.341210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.341207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15fda24-3fce-405b-b7ee-682fcfa242e6-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.341452 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.341310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4kg\" (UniqueName: \"kubernetes.io/projected/e15fda24-3fce-405b-b7ee-682fcfa242e6-kube-api-access-vx4kg\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.341592 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.341569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.341640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.341576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.343647 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.343628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15fda24-3fce-405b-b7ee-682fcfa242e6-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.349722 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.349700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4kg\" (UniqueName: \"kubernetes.io/projected/e15fda24-3fce-405b-b7ee-682fcfa242e6-kube-api-access-vx4kg\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.504306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.504278    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:31.641010 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:31.640987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"]
Jul 09 18:37:31.643352 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:37:31.643325    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15fda24_3fce_405b_b7ee_682fcfa242e6.slice/crio-2ee0992f8bf993afe351096a27e46f7285bd7e11a099662a55c7b709a8c2d680 WatchSource:0}: Error finding container 2ee0992f8bf993afe351096a27e46f7285bd7e11a099662a55c7b709a8c2d680: Status 404 returned error can't find the container with id 2ee0992f8bf993afe351096a27e46f7285bd7e11a099662a55c7b709a8c2d680
Jul 09 18:37:32.452112 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:32.452079    2581 generic.go:356] "Generic (PLEG): container finished" podID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerID="f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94" exitCode=0
Jul 09 18:37:32.452578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:32.452170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" event={"ID":"e15fda24-3fce-405b-b7ee-682fcfa242e6","Type":"ContainerDied","Data":"f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94"}
Jul 09 18:37:32.452578 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:32.452238    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" event={"ID":"e15fda24-3fce-405b-b7ee-682fcfa242e6","Type":"ContainerStarted","Data":"2ee0992f8bf993afe351096a27e46f7285bd7e11a099662a55c7b709a8c2d680"}
Jul 09 18:37:33.459098 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:33.459063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" event={"ID":"e15fda24-3fce-405b-b7ee-682fcfa242e6","Type":"ContainerStarted","Data":"7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"}
Jul 09 18:37:33.459521 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:33.459177    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:37:33.496881 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:33.496833    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" podStartSLOduration=1.86143876 podStartE2EDuration="2.496816303s" totalImagesPullingTime="0s" totalInitContainerRuntime="635.377543ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:37:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:37:33.495449979 +0000 UTC m=+2047.418994451" watchObservedRunningTime="2026-07-09 18:37:33.496816303 +0000 UTC m=+2047.420360775"
Jul 09 18:37:40.594101 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:40.594062    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:51.715063 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:51.714982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:37:51.784621 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:37:51.784589    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:38:04.464676 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:38:04.464644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:38:26.697660 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:38:26.697623    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:38:26.700142 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:38:26.700124    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:39:03.174834 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:39:03.174795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:39:03.238333 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:39:03.238297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:27.424152 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.424116    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"]
Jul 09 18:40:27.431557 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.431534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:27.433244 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:40:27.433216    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:40:27.433244 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:40:27.433244 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:40:27.433244 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba" cmd=["/bin/sleep","15"]
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.433266    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" message="" err=<
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:40:27.433282    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="main" containerID="cri-o://7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"
Jul 09 18:40:27.433479 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.433337    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="main" containerID="cri-o://7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba" gracePeriod=60
Jul 09 18:40:27.681571 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.681508    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:40:27.825351 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.825290    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-kserve-provision-location\") pod \"e15fda24-3fce-405b-b7ee-682fcfa242e6\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") "
Jul 09 18:40:27.825548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.825360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15fda24-3fce-405b-b7ee-682fcfa242e6-tls-certs\") pod \"e15fda24-3fce-405b-b7ee-682fcfa242e6\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") "
Jul 09 18:40:27.825548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.825438    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx4kg\" (UniqueName: \"kubernetes.io/projected/e15fda24-3fce-405b-b7ee-682fcfa242e6-kube-api-access-vx4kg\") pod \"e15fda24-3fce-405b-b7ee-682fcfa242e6\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") "
Jul 09 18:40:27.825548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.825468    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-tokenizer-uds\") pod \"e15fda24-3fce-405b-b7ee-682fcfa242e6\" (UID: \"e15fda24-3fce-405b-b7ee-682fcfa242e6\") "
Jul 09 18:40:27.825823 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.825791    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "e15fda24-3fce-405b-b7ee-682fcfa242e6" (UID: "e15fda24-3fce-405b-b7ee-682fcfa242e6"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:40:27.826077 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.826054    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e15fda24-3fce-405b-b7ee-682fcfa242e6" (UID: "e15fda24-3fce-405b-b7ee-682fcfa242e6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:40:27.827581 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.827551    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e15fda24-3fce-405b-b7ee-682fcfa242e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e15fda24-3fce-405b-b7ee-682fcfa242e6" (UID: "e15fda24-3fce-405b-b7ee-682fcfa242e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:40:27.827683 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.827557    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15fda24-3fce-405b-b7ee-682fcfa242e6-kube-api-access-vx4kg" (OuterVolumeSpecName: "kube-api-access-vx4kg") pod "e15fda24-3fce-405b-b7ee-682fcfa242e6" (UID: "e15fda24-3fce-405b-b7ee-682fcfa242e6"). InnerVolumeSpecName "kube-api-access-vx4kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:40:27.926666 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.926628    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:40:27.926666 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.926664    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e15fda24-3fce-405b-b7ee-682fcfa242e6-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:40:27.926855 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.926679    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx4kg\" (UniqueName: \"kubernetes.io/projected/e15fda24-3fce-405b-b7ee-682fcfa242e6-kube-api-access-vx4kg\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:40:27.926855 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:27.926692    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e15fda24-3fce-405b-b7ee-682fcfa242e6-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:40:28.297919 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.297889    2581 generic.go:356] "Generic (PLEG): container finished" podID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerID="7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba" exitCode=0
Jul 09 18:40:28.298094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.297945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" event={"ID":"e15fda24-3fce-405b-b7ee-682fcfa242e6","Type":"ContainerDied","Data":"7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"}
Jul 09 18:40:28.298094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.297950    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"
Jul 09 18:40:28.298094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.297985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226" event={"ID":"e15fda24-3fce-405b-b7ee-682fcfa242e6","Type":"ContainerDied","Data":"2ee0992f8bf993afe351096a27e46f7285bd7e11a099662a55c7b709a8c2d680"}
Jul 09 18:40:28.298094 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.298004    2581 scope.go:117] "RemoveContainer" containerID="7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"
Jul 09 18:40:28.307571 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.307549    2581 scope.go:117] "RemoveContainer" containerID="f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94"
Jul 09 18:40:28.315691 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.315673    2581 scope.go:117] "RemoveContainer" containerID="7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"
Jul 09 18:40:28.315933 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:40:28.315911    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba\": container with ID starting with 7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba not found: ID does not exist" containerID="7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"
Jul 09 18:40:28.316021 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.315946    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba"} err="failed to get container status \"7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba\": rpc error: code = NotFound desc = could not find container \"7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba\": container with ID starting with 7fbb9b0a08f0c464c4752d1c2cec831b2eadec687aa88545dda49cf76c2be3ba not found: ID does not exist"
Jul 09 18:40:28.316021 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.315969    2581 scope.go:117] "RemoveContainer" containerID="f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94"
Jul 09 18:40:28.316258 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:40:28.316232    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94\": container with ID starting with f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94 not found: ID does not exist" containerID="f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94"
Jul 09 18:40:28.316306 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.316266    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94"} err="failed to get container status \"f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94\": rpc error: code = NotFound desc = could not find container \"f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94\": container with ID starting with f2408275a6f4b2e5683f6d21807e4e136b5aa5ada8a84fb1b75342f1e1e52c94 not found: ID does not exist"
Jul 09 18:40:28.321851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.321828    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"]
Jul 09 18:40:28.325844 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.325824    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schegl226"]
Jul 09 18:40:28.800782 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.800749    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" path="/var/lib/kubelet/pods/e15fda24-3fce-405b-b7ee-682fcfa242e6/volumes"
Jul 09 18:40:28.898645 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:28.898616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:33.307100 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:33.307063    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:33.418608 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:33.418576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:50.113729 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:50.113643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:50.365245 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:50.365158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:40:54.428765 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:40:54.428733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:41:12.729289 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.729255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:41:12.937125 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.937092    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"]
Jul 09 18:41:12.937582 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.937565    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="storage-initializer"
Jul 09 18:41:12.937642 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.937585    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="storage-initializer"
Jul 09 18:41:12.937642 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.937605    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="main"
Jul 09 18:41:12.937642 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.937611    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="main"
Jul 09 18:41:12.937741 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.937678    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e15fda24-3fce-405b-b7ee-682fcfa242e6" containerName="main"
Jul 09 18:41:12.940919 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.940903    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:12.944404 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.944384    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:41:12.944848 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.944830    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:41:12.944927 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.944896    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-l825m\""
Jul 09 18:41:12.944978 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.944942    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:41:12.945033 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.944993    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:41:12.945033 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.945023    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 09 18:41:12.951954 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.951933    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"]
Jul 09 18:41:12.965308 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:12.965274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:41:13.021583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.021517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9w9k\" (UniqueName: \"kubernetes.io/projected/b7b1db82-e4b9-480f-9132-688bb42a8e38-kube-api-access-k9w9k\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.021583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.021551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.021583 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.021572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b1db82-e4b9-480f-9132-688bb42a8e38-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.021766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.021709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.122851 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.122820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.123002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.122898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9w9k\" (UniqueName: \"kubernetes.io/projected/b7b1db82-e4b9-480f-9132-688bb42a8e38-kube-api-access-k9w9k\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.123002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.122933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.123002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.122961    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b1db82-e4b9-480f-9132-688bb42a8e38-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.123325 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.123306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.123378 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.123350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.125405 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.125389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b1db82-e4b9-480f-9132-688bb42a8e38-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.131179 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.131155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9w9k\" (UniqueName: \"kubernetes.io/projected/b7b1db82-e4b9-480f-9132-688bb42a8e38-kube-api-access-k9w9k\") pod \"stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.251013 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.250985    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:13.386580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.386528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"]
Jul 09 18:41:13.389180 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:41:13.389155    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b1db82_e4b9_480f_9132_688bb42a8e38.slice/crio-5b3ad4b7c963e81a3bede7d2926c49022738ac92fd3544d5dbfe46ad3ee09668 WatchSource:0}: Error finding container 5b3ad4b7c963e81a3bede7d2926c49022738ac92fd3544d5dbfe46ad3ee09668: Status 404 returned error can't find the container with id 5b3ad4b7c963e81a3bede7d2926c49022738ac92fd3544d5dbfe46ad3ee09668
Jul 09 18:41:13.521718 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.521682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" event={"ID":"b7b1db82-e4b9-480f-9132-688bb42a8e38","Type":"ContainerStarted","Data":"3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e"}
Jul 09 18:41:13.521857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:13.521737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" event={"ID":"b7b1db82-e4b9-480f-9132-688bb42a8e38","Type":"ContainerStarted","Data":"5b3ad4b7c963e81a3bede7d2926c49022738ac92fd3544d5dbfe46ad3ee09668"}
Jul 09 18:41:14.528176 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:14.528142    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerID="3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e" exitCode=0
Jul 09 18:41:14.528576 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:14.528228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" event={"ID":"b7b1db82-e4b9-480f-9132-688bb42a8e38","Type":"ContainerDied","Data":"3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e"}
Jul 09 18:41:15.534805 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:15.534770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" event={"ID":"b7b1db82-e4b9-480f-9132-688bb42a8e38","Type":"ContainerStarted","Data":"18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"}
Jul 09 18:41:15.535172 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:15.534901    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:41:15.561998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:15.561953    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" podStartSLOduration=2.927552013 podStartE2EDuration="3.561937673s" totalImagesPullingTime="0s" totalInitContainerRuntime="634.38566ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:41:15.560897176 +0000 UTC m=+2269.484441648" watchObservedRunningTime="2026-07-09 18:41:15.561937673 +0000 UTC m=+2269.485482145"
Jul 09 18:41:26.654632 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:26.654597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:41:46.541891 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:41:46.541858    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:43:26.727822 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:43:26.727796    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:43:26.731793 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:43:26.731772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:44:16.294504 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:16.294414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:16.374049 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:16.374017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:24.298747 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.298695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:24.365558 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.365522    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"]
Jul 09 18:44:24.375004 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:44:24.374968    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:44:24.375004 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:44:24.375004 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:44:24.375004 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42" cmd=["/bin/sleep","15"]
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.375021    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" message="" err=<
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:44:24.375036    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="main" containerID="cri-o://18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"
Jul 09 18:44:24.375353 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.375080    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="main" containerID="cri-o://18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42" gracePeriod=60
Jul 09 18:44:24.404611 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.404583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:24.469150 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.469116    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:24.631748 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.631727    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:44:24.803536 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.803510    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b1db82-e4b9-480f-9132-688bb42a8e38-tls-certs\") pod \"b7b1db82-e4b9-480f-9132-688bb42a8e38\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") "
Jul 09 18:44:24.803717 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.803549    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-tokenizer-uds\") pod \"b7b1db82-e4b9-480f-9132-688bb42a8e38\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") "
Jul 09 18:44:24.803717 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.803577    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-kserve-provision-location\") pod \"b7b1db82-e4b9-480f-9132-688bb42a8e38\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") "
Jul 09 18:44:24.803717 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.803599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9w9k\" (UniqueName: \"kubernetes.io/projected/b7b1db82-e4b9-480f-9132-688bb42a8e38-kube-api-access-k9w9k\") pod \"b7b1db82-e4b9-480f-9132-688bb42a8e38\" (UID: \"b7b1db82-e4b9-480f-9132-688bb42a8e38\") "
Jul 09 18:44:24.803883 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.803826    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b7b1db82-e4b9-480f-9132-688bb42a8e38" (UID: "b7b1db82-e4b9-480f-9132-688bb42a8e38"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:44:24.804228 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.804176    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b7b1db82-e4b9-480f-9132-688bb42a8e38" (UID: "b7b1db82-e4b9-480f-9132-688bb42a8e38"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:44:24.805728 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.805702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7b1db82-e4b9-480f-9132-688bb42a8e38-kube-api-access-k9w9k" (OuterVolumeSpecName: "kube-api-access-k9w9k") pod "b7b1db82-e4b9-480f-9132-688bb42a8e38" (UID: "b7b1db82-e4b9-480f-9132-688bb42a8e38"). InnerVolumeSpecName "kube-api-access-k9w9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:44:24.805832 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.805796    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7b1db82-e4b9-480f-9132-688bb42a8e38-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7b1db82-e4b9-480f-9132-688bb42a8e38" (UID: "b7b1db82-e4b9-480f-9132-688bb42a8e38"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:44:24.904947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.904866    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b1db82-e4b9-480f-9132-688bb42a8e38-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:44:24.904947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.904894    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:44:24.904947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.904903    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7b1db82-e4b9-480f-9132-688bb42a8e38-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:44:24.904947 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:24.904912    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9w9k\" (UniqueName: \"kubernetes.io/projected/b7b1db82-e4b9-480f-9132-688bb42a8e38-kube-api-access-k9w9k\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:44:25.462400 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.462368    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerID="18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42" exitCode=0
Jul 09 18:44:25.462786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.462418    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"
Jul 09 18:44:25.462786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.462455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" event={"ID":"b7b1db82-e4b9-480f-9132-688bb42a8e38","Type":"ContainerDied","Data":"18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"}
Jul 09 18:44:25.462786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.462507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8" event={"ID":"b7b1db82-e4b9-480f-9132-688bb42a8e38","Type":"ContainerDied","Data":"5b3ad4b7c963e81a3bede7d2926c49022738ac92fd3544d5dbfe46ad3ee09668"}
Jul 09 18:44:25.462786 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.462528    2581 scope.go:117] "RemoveContainer" containerID="18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"
Jul 09 18:44:25.473893 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.473866    2581 scope.go:117] "RemoveContainer" containerID="3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e"
Jul 09 18:44:25.481790 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.481758    2581 scope.go:117] "RemoveContainer" containerID="18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"
Jul 09 18:44:25.482032 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:44:25.482010    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42\": container with ID starting with 18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42 not found: ID does not exist" containerID="18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"
Jul 09 18:44:25.482123 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.482042    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42"} err="failed to get container status \"18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42\": rpc error: code = NotFound desc = could not find container \"18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42\": container with ID starting with 18495d4085016776bc1c059e25c28702593f8898ea6d5f63331d47b08867ca42 not found: ID does not exist"
Jul 09 18:44:25.482123 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.482066    2581 scope.go:117] "RemoveContainer" containerID="3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e"
Jul 09 18:44:25.482360 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:44:25.482343    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e\": container with ID starting with 3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e not found: ID does not exist" containerID="3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e"
Jul 09 18:44:25.482401 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.482365    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e"} err="failed to get container status \"3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e\": rpc error: code = NotFound desc = could not find container \"3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e\": container with ID starting with 3123f2c2a86ac89841fef66a4a0d0ba83cddb6e32807e21a36f7feb77da5bf2e not found: ID does not exist"
Jul 09 18:44:25.489080 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.489060    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"]
Jul 09 18:44:25.493599 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:25.493572    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-55ff964456-f7kl8"]
Jul 09 18:44:26.800561 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:26.800528    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" path="/var/lib/kubelet/pods/b7b1db82-e4b9-480f-9132-688bb42a8e38/volumes"
Jul 09 18:44:37.161761 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:37.161719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:37.261279 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:37.261247    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:40.977788 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:40.977748    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:44:55.664927 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:44:55.664891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:04.232411 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:04.232375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:04.546991 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:04.546898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:14.106886 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:14.106847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:26.803695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:26.803653    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:27.370995 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.370960    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"]
Jul 09 18:45:27.371380 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.371366    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="main"
Jul 09 18:45:27.371436 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.371382    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="main"
Jul 09 18:45:27.371436 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.371410    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="storage-initializer"
Jul 09 18:45:27.371436 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.371417    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="storage-initializer"
Jul 09 18:45:27.371548 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.371473    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7b1db82-e4b9-480f-9132-688bb42a8e38" containerName="main"
Jul 09 18:45:27.374832 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.374810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.377340 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.377321    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:45:27.378674 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.378656    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-epp-sa-dockercfg-82xbp\""
Jul 09 18:45:27.379257 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.379237    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:45:27.379351 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.379261    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:45:27.379351 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.379275    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Jul 09 18:45:27.379351 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.379278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:45:27.386634 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.386605    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"]
Jul 09 18:45:27.477694 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.477567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:27.556316 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.556280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.556470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.556339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.556470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.556361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qslqg\" (UniqueName: \"kubernetes.io/projected/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kube-api-access-qslqg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.556470 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.556434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.574478 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.574444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:45:27.657646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.657566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.657778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.657641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.657778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.657699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.657778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.657725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qslqg\" (UniqueName: \"kubernetes.io/projected/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kube-api-access-qslqg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.658098 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.658074    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.658098 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.658088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.659990 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.659972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.666788 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.666767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qslqg\" (UniqueName: \"kubernetes.io/projected/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kube-api-access-qslqg\") pod \"router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.684587 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.684551    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:27.820818 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:27.820791    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"]
Jul 09 18:45:27.823367 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:45:27.823341    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6d0bcf_0de5_4c4e_a831_c8283b6a4562.slice/crio-12ec2d00108b6a80d06ec6962543285e23f49bd4514ef4c567fbadfafb400929 WatchSource:0}: Error finding container 12ec2d00108b6a80d06ec6962543285e23f49bd4514ef4c567fbadfafb400929: Status 404 returned error can't find the container with id 12ec2d00108b6a80d06ec6962543285e23f49bd4514ef4c567fbadfafb400929
Jul 09 18:45:28.768575 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:28.768535    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerID="72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf" exitCode=0
Jul 09 18:45:28.768732 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:28.768613    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" event={"ID":"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562","Type":"ContainerDied","Data":"72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf"}
Jul 09 18:45:28.768732 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:28.768659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" event={"ID":"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562","Type":"ContainerStarted","Data":"12ec2d00108b6a80d06ec6962543285e23f49bd4514ef4c567fbadfafb400929"}
Jul 09 18:45:29.775588 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:29.775544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" event={"ID":"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562","Type":"ContainerStarted","Data":"af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"}
Jul 09 18:45:29.776014 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:29.775612    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:45:29.797432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:29.797382    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" podStartSLOduration=2.138668223 podStartE2EDuration="2.797367463s" totalImagesPullingTime="0s" totalInitContainerRuntime="658.69924ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:45:29.7954391 +0000 UTC m=+2523.718983592" watchObservedRunningTime="2026-07-09 18:45:29.797367463 +0000 UTC m=+2523.720911934"
Jul 09 18:45:38.990862 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:45:38.990825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:46:00.782036 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:46:00.782004    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:46:41.667501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:46:41.667468    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:47:45.752479 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:47:45.752439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:47:45.846083 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:47:45.846050    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:48:14.904588 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:14.904544    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:48:26.759229 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:26.759134    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:48:26.764226 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:26.764204    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:48:36.559793 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.559759    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:48:36.572210 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.572163    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"]
Jul 09 18:48:36.581570 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:48:36.581533    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 18:48:36.581570 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:48:36.581570 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:48:36.581570 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe" cmd=["/bin/sleep","15"]
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.581586    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" message="" err=<
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:48:36.581598    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="main" containerID="cri-o://af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"
Jul 09 18:48:36.581837 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.581644    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="main" containerID="cri-o://af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe" gracePeriod=60
Jul 09 18:48:36.638646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.638612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:48:36.838912 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.838886    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:48:36.872549 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.872503    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tokenizer-uds\") pod \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") "
Jul 09 18:48:36.872740 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.872557    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kserve-provision-location\") pod \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") "
Jul 09 18:48:36.872740 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.872591    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qslqg\" (UniqueName: \"kubernetes.io/projected/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kube-api-access-qslqg\") pod \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") "
Jul 09 18:48:36.872740 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.872692    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tls-certs\") pod \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\" (UID: \"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562\") "
Jul 09 18:48:36.872883 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.872762    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" (UID: "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:48:36.873002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.872978    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:48:36.873429 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.873396    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" (UID: "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:48:36.874841 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.874816    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" (UID: "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:48:36.874970 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.874920    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kube-api-access-qslqg" (OuterVolumeSpecName: "kube-api-access-qslqg") pod "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" (UID: "dd6d0bcf-0de5-4c4e-a831-c8283b6a4562"). InnerVolumeSpecName "kube-api-access-qslqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:48:36.974147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.974111    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:48:36.974147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.974143    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:48:36.974147 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:36.974153    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qslqg\" (UniqueName: \"kubernetes.io/projected/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562-kube-api-access-qslqg\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:48:37.685013 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.684972    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerID="af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe" exitCode=0
Jul 09 18:48:37.685462 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.685031    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"
Jul 09 18:48:37.685462 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.685060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" event={"ID":"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562","Type":"ContainerDied","Data":"af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"}
Jul 09 18:48:37.685462 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.685115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6" event={"ID":"dd6d0bcf-0de5-4c4e-a831-c8283b6a4562","Type":"ContainerDied","Data":"12ec2d00108b6a80d06ec6962543285e23f49bd4514ef4c567fbadfafb400929"}
Jul 09 18:48:37.685462 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.685136    2581 scope.go:117] "RemoveContainer" containerID="af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"
Jul 09 18:48:37.694475 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.694458    2581 scope.go:117] "RemoveContainer" containerID="72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf"
Jul 09 18:48:37.702369 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.702351    2581 scope.go:117] "RemoveContainer" containerID="af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"
Jul 09 18:48:37.702633 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:48:37.702616    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe\": container with ID starting with af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe not found: ID does not exist" containerID="af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"
Jul 09 18:48:37.702701 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.702640    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe"} err="failed to get container status \"af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe\": rpc error: code = NotFound desc = could not find container \"af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe\": container with ID starting with af8b368fad2e185c937b89cd4c68c30c4c9310fbf5710d38de3e0799c1f675fe not found: ID does not exist"
Jul 09 18:48:37.702701 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.702657    2581 scope.go:117] "RemoveContainer" containerID="72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf"
Jul 09 18:48:37.702871 ip-10-0-143-79 kubenswrapper[2581]: E0709 18:48:37.702856    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf\": container with ID starting with 72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf not found: ID does not exist" containerID="72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf"
Jul 09 18:48:37.702908 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.702875    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf"} err="failed to get container status \"72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf\": rpc error: code = NotFound desc = could not find container \"72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf\": container with ID starting with 72df8c9acc31693d7fad31d2f83776c62a947ac42ae47a7cb47273d791ae47cf not found: ID does not exist"
Jul 09 18:48:37.708799 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.708772    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"]
Jul 09 18:48:37.714062 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:37.714040    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5fdb8f69cpvwr6"]
Jul 09 18:48:38.800231 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:38.800178    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" path="/var/lib/kubelet/pods/dd6d0bcf-0de5-4c4e-a831-c8283b6a4562/volumes"
Jul 09 18:48:45.468149 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:48:45.468118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:49:00.845439 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:49:00.845406    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:50:23.271133 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:50:23.271088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:50:27.686170 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:50:27.686126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:50:45.568024 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:50:45.567980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:50:58.675243 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:50:58.675205    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:51:28.278911 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:28.278840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:51:53.320264 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.320223    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:51:53.481449 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.481417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:51:53.576501 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.576420    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:51:53.586206 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.586158    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 09 18:51:53.586570 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.586557    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="main"
Jul 09 18:51:53.586612 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.586572    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="main"
Jul 09 18:51:53.586612 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.586588    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="storage-initializer"
Jul 09 18:51:53.586612 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.586593    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="storage-initializer"
Jul 09 18:51:53.586744 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.586645    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd6d0bcf-0de5-4c4e-a831-c8283b6a4562" containerName="main"
Jul 09 18:51:53.589918 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.589901    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.592661 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.592635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Jul 09 18:51:53.592661 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.592651    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:51:53.593950 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.593929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 18:51:53.594035 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.593964    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:51:53.594035 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.593977    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-4l8zc\""
Jul 09 18:51:53.594035 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.593963    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:51:53.601539 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.601516    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 09 18:51:53.616121 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.616301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.616301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.616301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.616432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.616432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzvtd\" (UniqueName: \"kubernetes.io/projected/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kube-api-access-nzvtd\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.616432 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.616352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717703 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717703 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzvtd\" (UniqueName: \"kubernetes.io/projected/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kube-api-access-nzvtd\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.717953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.717858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.718239 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.718052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.718239 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.718118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.718239 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.718217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.718383 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.718247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.720130 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.720106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.720333 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.720315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.731305 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.731261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzvtd\" (UniqueName: \"kubernetes.io/projected/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kube-api-access-nzvtd\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:53.787509 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.787477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:51:53.901719 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:53.901687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:51:54.045408 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:54.045380    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 09 18:51:54.046933 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:51:54.046908    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7bfa2e_a7f2_4484_bdc0_ae4800fdaec2.slice/crio-5159c7da81dd4e8b062ad9b48e62614cab9ebfafadb03cd9f19fc1abe92844a9 WatchSource:0}: Error finding container 5159c7da81dd4e8b062ad9b48e62614cab9ebfafadb03cd9f19fc1abe92844a9: Status 404 returned error can't find the container with id 5159c7da81dd4e8b062ad9b48e62614cab9ebfafadb03cd9f19fc1abe92844a9
Jul 09 18:51:54.635596 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:54.635562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2","Type":"ContainerStarted","Data":"dafeb0a6e7062bd0e7ed1a104e9c51eee692f2fd184533d3bcbe285480bd8245"}
Jul 09 18:51:54.635977 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:54.635616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2","Type":"ContainerStarted","Data":"5159c7da81dd4e8b062ad9b48e62614cab9ebfafadb03cd9f19fc1abe92844a9"}
Jul 09 18:51:57.652998 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:57.652959    2581 generic.go:356] "Generic (PLEG): container finished" podID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerID="dafeb0a6e7062bd0e7ed1a104e9c51eee692f2fd184533d3bcbe285480bd8245" exitCode=0
Jul 09 18:51:57.653403 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:57.653037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2","Type":"ContainerDied","Data":"dafeb0a6e7062bd0e7ed1a104e9c51eee692f2fd184533d3bcbe285480bd8245"}
Jul 09 18:51:57.654441 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:51:57.654423    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:52:00.331822 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:52:00.331781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:52:42.910049 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:52:42.910013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2","Type":"ContainerStarted","Data":"461fb42efa5f54f5adf71def564de70bf6a48a486bcd9cc1fe4c7c84124316b8"}
Jul 09 18:52:42.940079 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:52:42.932901    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" podStartSLOduration=1.844610459 podStartE2EDuration="49.932879391s" totalImagesPullingTime="45.119964507s" totalInitContainerRuntime="2.968304425s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:52:42.93086251 +0000 UTC m=+2956.854406982" watchObservedRunningTime="2026-07-09 18:52:42.932879391 +0000 UTC m=+2956.856423864"
Jul 09 18:52:52.317996 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:52:52.317911    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:52:52.421301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:52:52.421264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:53:19.340457 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:53:19.340423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:53:26.795389 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:53:26.795355    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:53:26.803404 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:53:26.803378    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:53:48.522174 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:53:48.522144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:53:48.811057 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:53:48.810955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:54:03.418686 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:54:03.418643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:24.804776 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:55:24.804735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:54.246441 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:55:54.246363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:55:56.020553 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:55:56.020525    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:00.294722 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:00.294687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:00.390530 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:00.390495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:06.496415 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:06.496377    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:06.648331 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:06.648292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:16.385966 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:16.385926    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 09 18:56:26.933611 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:26.933575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:27.275495 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:27.275402    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:31.408803 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:31.408739    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerName="main" containerID="cri-o://461fb42efa5f54f5adf71def564de70bf6a48a486bcd9cc1fe4c7c84124316b8" gracePeriod=45
Jul 09 18:56:32.043949 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.043915    2581 generic.go:356] "Generic (PLEG): container finished" podID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerID="461fb42efa5f54f5adf71def564de70bf6a48a486bcd9cc1fe4c7c84124316b8" exitCode=0
Jul 09 18:56:32.044098 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.043963    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2","Type":"ContainerDied","Data":"461fb42efa5f54f5adf71def564de70bf6a48a486bcd9cc1fe4c7c84124316b8"}
Jul 09 18:56:32.088798 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.088774    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:56:32.165857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.165768    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-home\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.165857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.165807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzvtd\" (UniqueName: \"kubernetes.io/projected/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kube-api-access-nzvtd\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.165857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.165828    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-model-cache\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.165857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.165842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-dshm\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.166233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.165912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tls-certs\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.166233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.165938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tmp-dir\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.166233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.166041    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kserve-provision-location\") pod \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\" (UID: \"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2\") "
Jul 09 18:56:32.166233 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.166128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:32.166459 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.166402    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:32.166670 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.166639    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-home" (OuterVolumeSpecName: "home") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:32.168195 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.168160    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:56:32.168511 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.168488    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kube-api-access-nzvtd" (OuterVolumeSpecName: "kube-api-access-nzvtd") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "kube-api-access-nzvtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:56:32.168616 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.168493    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-dshm" (OuterVolumeSpecName: "dshm") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:32.184072 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.184037    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:32.226002 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.225962    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" (UID: "8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:56:32.266987 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.266953    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kserve-provision-location\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:32.266987 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.266984    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:32.267160 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.266996    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzvtd\" (UniqueName: \"kubernetes.io/projected/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-kube-api-access-nzvtd\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:32.267160 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.267005    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:32.267160 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.267013    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:32.267160 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:32.267020    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:56:33.050639 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:33.050610    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Jul 09 18:56:33.051013 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:33.050608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2","Type":"ContainerDied","Data":"5159c7da81dd4e8b062ad9b48e62614cab9ebfafadb03cd9f19fc1abe92844a9"}
Jul 09 18:56:33.051013 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:33.050750    2581 scope.go:117] "RemoveContainer" containerID="461fb42efa5f54f5adf71def564de70bf6a48a486bcd9cc1fe4c7c84124316b8"
Jul 09 18:56:33.059410 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:33.059384    2581 scope.go:117] "RemoveContainer" containerID="dafeb0a6e7062bd0e7ed1a104e9c51eee692f2fd184533d3bcbe285480bd8245"
Jul 09 18:56:33.071857 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:33.071832    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 09 18:56:33.076121 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:33.076097    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Jul 09 18:56:34.801171 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:34.801126    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" path="/var/lib/kubelet/pods/8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2/volumes"
Jul 09 18:56:53.518171 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:53.518134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:56.279337 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.279309    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"]
Jul 09 18:56:56.279727 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.279711    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerName="storage-initializer"
Jul 09 18:56:56.279769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.279729    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerName="storage-initializer"
Jul 09 18:56:56.279769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.279739    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerName="main"
Jul 09 18:56:56.279769 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.279745    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerName="main"
Jul 09 18:56:56.279865 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.279814    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7bfa2e-a7f2-4484-bdc0-ae4800fdaec2" containerName="main"
Jul 09 18:56:56.283108 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.283086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.285703 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.285683    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:56:56.285820 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.285683    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:56:56.286651 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.286633    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:56:56.286766 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.286667    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jul 09 18:56:56.294503 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.294481    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"]
Jul 09 18:56:56.310854 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.310829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:56.383820 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.383792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.383820 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.383821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.384031 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.383845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.384031 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.383862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx5zv\" (UniqueName: \"kubernetes.io/projected/829a4e6d-e298-41cb-bb45-9f26b28ec532-kube-api-access-vx5zv\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.384031 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.383959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.384031 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.383985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/829a4e6d-e298-41cb-bb45-9f26b28ec532-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484440 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484440 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx5zv\" (UniqueName: \"kubernetes.io/projected/829a4e6d-e298-41cb-bb45-9f26b28ec532-kube-api-access-vx5zv\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/829a4e6d-e298-41cb-bb45-9f26b28ec532-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484903 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.484953 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.484900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.485305 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.485281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.486862 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.486843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.487165 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.487148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/829a4e6d-e298-41cb-bb45-9f26b28ec532-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.496703 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.496681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx5zv\" (UniqueName: \"kubernetes.io/projected/829a4e6d-e298-41cb-bb45-9f26b28ec532-kube-api-access-vx5zv\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.593113 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.593031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:56:56.697033 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.696995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:56:56.735809 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:56.735778    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"]
Jul 09 18:56:56.738877 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:56:56.738852    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829a4e6d_e298_41cb_bb45_9f26b28ec532.slice/crio-993caa2febd5964aca478fe3fd533b6ccc8653e9e21a8a8b41d553c7aee82ded WatchSource:0}: Error finding container 993caa2febd5964aca478fe3fd533b6ccc8653e9e21a8a8b41d553c7aee82ded: Status 404 returned error can't find the container with id 993caa2febd5964aca478fe3fd533b6ccc8653e9e21a8a8b41d553c7aee82ded
Jul 09 18:56:57.165139 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:57.165098    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8" event={"ID":"829a4e6d-e298-41cb-bb45-9f26b28ec532","Type":"ContainerStarted","Data":"f637af39ad5ac389afe23a4d87bb52d4d0724bcab6dc1b8ee4b63f1380a7c0d1"}
Jul 09 18:56:57.165404 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:57.165158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8" event={"ID":"829a4e6d-e298-41cb-bb45-9f26b28ec532","Type":"ContainerStarted","Data":"993caa2febd5964aca478fe3fd533b6ccc8653e9e21a8a8b41d553c7aee82ded"}
Jul 09 18:56:57.214712 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:57.214663    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8" podStartSLOduration=1.2146458 podStartE2EDuration="1.2146458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:56:57.214059569 +0000 UTC m=+3211.137604041" watchObservedRunningTime="2026-07-09 18:56:57.2146458 +0000 UTC m=+3211.138190272"
Jul 09 18:56:57.593999 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:56:57.593912    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:57:04.090973 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:04.090935    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:57:06.593162 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:06.593128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:57:06.605614 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:06.605589    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:57:07.231315 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:07.231278    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:57:20.317364 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:20.317278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:57:30.222472 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:30.222421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:57:30.247268 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:30.247240    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"]
Jul 09 18:57:30.297208 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:30.297167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:57:41.264544 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:41.264506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:57:45.267638 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.267600    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8" podUID="829a4e6d-e298-41cb-bb45-9f26b28ec532" containerName="main" containerID="cri-o://f637af39ad5ac389afe23a4d87bb52d4d0724bcab6dc1b8ee4b63f1380a7c0d1" gracePeriod=45
Jul 09 18:57:45.405627 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.405591    2581 generic.go:356] "Generic (PLEG): container finished" podID="829a4e6d-e298-41cb-bb45-9f26b28ec532" containerID="f637af39ad5ac389afe23a4d87bb52d4d0724bcab6dc1b8ee4b63f1380a7c0d1" exitCode=0
Jul 09 18:57:45.405785 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.405648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8" event={"ID":"829a4e6d-e298-41cb-bb45-9f26b28ec532","Type":"ContainerDied","Data":"f637af39ad5ac389afe23a4d87bb52d4d0724bcab6dc1b8ee4b63f1380a7c0d1"}
Jul 09 18:57:45.520770 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.520713    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:57:45.625020 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.624991    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-model-cache\") pod \"829a4e6d-e298-41cb-bb45-9f26b28ec532\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") "
Jul 09 18:57:45.625020 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625024    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-home\") pod \"829a4e6d-e298-41cb-bb45-9f26b28ec532\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") "
Jul 09 18:57:45.625301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-dshm\") pod \"829a4e6d-e298-41cb-bb45-9f26b28ec532\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") "
Jul 09 18:57:45.625301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625097    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx5zv\" (UniqueName: \"kubernetes.io/projected/829a4e6d-e298-41cb-bb45-9f26b28ec532-kube-api-access-vx5zv\") pod \"829a4e6d-e298-41cb-bb45-9f26b28ec532\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") "
Jul 09 18:57:45.625301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-tmp-dir\") pod \"829a4e6d-e298-41cb-bb45-9f26b28ec532\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") "
Jul 09 18:57:45.625301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/829a4e6d-e298-41cb-bb45-9f26b28ec532-tls-certs\") pod \"829a4e6d-e298-41cb-bb45-9f26b28ec532\" (UID: \"829a4e6d-e298-41cb-bb45-9f26b28ec532\") "
Jul 09 18:57:45.625301 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-model-cache" (OuterVolumeSpecName: "model-cache") pod "829a4e6d-e298-41cb-bb45-9f26b28ec532" (UID: "829a4e6d-e298-41cb-bb45-9f26b28ec532"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:57:45.625552 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-home" (OuterVolumeSpecName: "home") pod "829a4e6d-e298-41cb-bb45-9f26b28ec532" (UID: "829a4e6d-e298-41cb-bb45-9f26b28ec532"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:57:45.625552 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625446    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:57:45.625552 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.625435    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "829a4e6d-e298-41cb-bb45-9f26b28ec532" (UID: "829a4e6d-e298-41cb-bb45-9f26b28ec532"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:57:45.627211 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.627164    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-dshm" (OuterVolumeSpecName: "dshm") pod "829a4e6d-e298-41cb-bb45-9f26b28ec532" (UID: "829a4e6d-e298-41cb-bb45-9f26b28ec532"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:57:45.627319 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.627242    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829a4e6d-e298-41cb-bb45-9f26b28ec532-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "829a4e6d-e298-41cb-bb45-9f26b28ec532" (UID: "829a4e6d-e298-41cb-bb45-9f26b28ec532"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:57:45.627319 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.627268    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829a4e6d-e298-41cb-bb45-9f26b28ec532-kube-api-access-vx5zv" (OuterVolumeSpecName: "kube-api-access-vx5zv") pod "829a4e6d-e298-41cb-bb45-9f26b28ec532" (UID: "829a4e6d-e298-41cb-bb45-9f26b28ec532"). InnerVolumeSpecName "kube-api-access-vx5zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:57:45.726872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.726834    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/829a4e6d-e298-41cb-bb45-9f26b28ec532-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:57:45.726872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.726865    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:57:45.726872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.726873    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:57:45.726872 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.726882    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx5zv\" (UniqueName: \"kubernetes.io/projected/829a4e6d-e298-41cb-bb45-9f26b28ec532-kube-api-access-vx5zv\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:57:45.727125 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:45.726890    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/829a4e6d-e298-41cb-bb45-9f26b28ec532-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:57:46.412307 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:46.412274    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"
Jul 09 18:57:46.412307 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:46.412292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8" event={"ID":"829a4e6d-e298-41cb-bb45-9f26b28ec532","Type":"ContainerDied","Data":"993caa2febd5964aca478fe3fd533b6ccc8653e9e21a8a8b41d553c7aee82ded"}
Jul 09 18:57:46.412801 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:46.412379    2581 scope.go:117] "RemoveContainer" containerID="f637af39ad5ac389afe23a4d87bb52d4d0724bcab6dc1b8ee4b63f1380a7c0d1"
Jul 09 18:57:46.435252 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:46.435223    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"]
Jul 09 18:57:46.439530 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:46.439508    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-9nvf8"]
Jul 09 18:57:46.801003 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:57:46.800926    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="829a4e6d-e298-41cb-bb45-9f26b28ec532" path="/var/lib/kubelet/pods/829a4e6d-e298-41cb-bb45-9f26b28ec532/volumes"
Jul 09 18:58:01.984924 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.984890    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"]
Jul 09 18:58:01.985394 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.985306    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829a4e6d-e298-41cb-bb45-9f26b28ec532" containerName="main"
Jul 09 18:58:01.985394 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.985318    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="829a4e6d-e298-41cb-bb45-9f26b28ec532" containerName="main"
Jul 09 18:58:01.985394 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.985376    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="829a4e6d-e298-41cb-bb45-9f26b28ec532" containerName="main"
Jul 09 18:58:01.989732 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.989713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:01.993359 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.993322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:58:01.993359 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.993322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Jul 09 18:58:01.993524 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.993447    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:58:01.993524 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.993496    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:58:01.998499 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:01.998476    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"]
Jul 09 18:58:02.011629 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.011598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:02.067480 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.067448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.067646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.067485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.067646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.067518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029756ea-96dc-4a43-b349-a00139d7e220-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.067646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.067588    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.067809 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.067683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn5vr\" (UniqueName: \"kubernetes.io/projected/029756ea-96dc-4a43-b349-a00139d7e220-kube-api-access-dn5vr\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.067809 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.067758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169196 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029756ea-96dc-4a43-b349-a00139d7e220-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169367 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn5vr\" (UniqueName: \"kubernetes.io/projected/029756ea-96dc-4a43-b349-a00139d7e220-kube-api-access-dn5vr\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169643 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169619    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169715 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.169836 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.169814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.171658 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.171630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.171812 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.171797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029756ea-96dc-4a43-b349-a00139d7e220-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.177641 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.177620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn5vr\" (UniqueName: \"kubernetes.io/projected/029756ea-96dc-4a43-b349-a00139d7e220-kube-api-access-dn5vr\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.262389 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.262306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:02.300460 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.300433    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:02.434988 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.434964    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"]
Jul 09 18:58:02.437685 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:58:02.437656    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029756ea_96dc_4a43_b349_a00139d7e220.slice/crio-934b4d5fa465c684760772bb6f3b00e8bcef3d27278a730dcea6adba50836ab6 WatchSource:0}: Error finding container 934b4d5fa465c684760772bb6f3b00e8bcef3d27278a730dcea6adba50836ab6: Status 404 returned error can't find the container with id 934b4d5fa465c684760772bb6f3b00e8bcef3d27278a730dcea6adba50836ab6
Jul 09 18:58:02.493000 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:02.492975    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc" event={"ID":"029756ea-96dc-4a43-b349-a00139d7e220","Type":"ContainerStarted","Data":"934b4d5fa465c684760772bb6f3b00e8bcef3d27278a730dcea6adba50836ab6"}
Jul 09 18:58:03.499771 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:03.499733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc" event={"ID":"029756ea-96dc-4a43-b349-a00139d7e220","Type":"ContainerStarted","Data":"eafaf761dcb1a18d9c4f86a0c645f39fcfdbbfe929e271eb0048281240e58ef5"}
Jul 09 18:58:04.301182 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:04.301144    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:12.300636 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:12.300591    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:12.312701 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:12.312672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:12.332155 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:12.332103    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc" podStartSLOduration=11.332089948 podStartE2EDuration="11.332089948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:58:03.517824509 +0000 UTC m=+3277.441368983" watchObservedRunningTime="2026-07-09 18:58:12.332089948 +0000 UTC m=+3286.255634419"
Jul 09 18:58:12.556000 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:12.555921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:58:16.216534 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:16.216494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:26.830312 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:26.830281    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:58:26.840081 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:26.840060    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 18:58:44.733819 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:44.733765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:45.525309 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:45.525273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:45.540216 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:45.540173    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"]
Jul 09 18:58:45.601074 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:45.601043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:48.639007 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:48.638924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:58:48.734981 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:58:48.734946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:59:00.560404 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.560354    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc" podUID="029756ea-96dc-4a43-b349-a00139d7e220" containerName="main" containerID="cri-o://eafaf761dcb1a18d9c4f86a0c645f39fcfdbbfe929e271eb0048281240e58ef5" gracePeriod=45
Jul 09 18:59:00.782087 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.782053    2581 generic.go:356] "Generic (PLEG): container finished" podID="029756ea-96dc-4a43-b349-a00139d7e220" containerID="eafaf761dcb1a18d9c4f86a0c645f39fcfdbbfe929e271eb0048281240e58ef5" exitCode=0
Jul 09 18:59:00.782286 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.782124    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc" event={"ID":"029756ea-96dc-4a43-b349-a00139d7e220","Type":"ContainerDied","Data":"eafaf761dcb1a18d9c4f86a0c645f39fcfdbbfe929e271eb0048281240e58ef5"}
Jul 09 18:59:00.805922 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.805903    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:59:00.890634 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-dshm\") pod \"029756ea-96dc-4a43-b349-a00139d7e220\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") "
Jul 09 18:59:00.890778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890698    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-model-cache\") pod \"029756ea-96dc-4a43-b349-a00139d7e220\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") "
Jul 09 18:59:00.890778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890718    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029756ea-96dc-4a43-b349-a00139d7e220-tls-certs\") pod \"029756ea-96dc-4a43-b349-a00139d7e220\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") "
Jul 09 18:59:00.890778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890737    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-tmp-dir\") pod \"029756ea-96dc-4a43-b349-a00139d7e220\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") "
Jul 09 18:59:00.890778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn5vr\" (UniqueName: \"kubernetes.io/projected/029756ea-96dc-4a43-b349-a00139d7e220-kube-api-access-dn5vr\") pod \"029756ea-96dc-4a43-b349-a00139d7e220\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") "
Jul 09 18:59:00.890984 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890790    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-home\") pod \"029756ea-96dc-4a43-b349-a00139d7e220\" (UID: \"029756ea-96dc-4a43-b349-a00139d7e220\") "
Jul 09 18:59:00.891056 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.890994    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "029756ea-96dc-4a43-b349-a00139d7e220" (UID: "029756ea-96dc-4a43-b349-a00139d7e220"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:00.891125 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.891019    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-model-cache" (OuterVolumeSpecName: "model-cache") pod "029756ea-96dc-4a43-b349-a00139d7e220" (UID: "029756ea-96dc-4a43-b349-a00139d7e220"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:00.891418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.891364    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:00.891418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.891386    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:00.891798 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.891767    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-home" (OuterVolumeSpecName: "home") pod "029756ea-96dc-4a43-b349-a00139d7e220" (UID: "029756ea-96dc-4a43-b349-a00139d7e220"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:00.892920 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.892897    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-dshm" (OuterVolumeSpecName: "dshm") pod "029756ea-96dc-4a43-b349-a00139d7e220" (UID: "029756ea-96dc-4a43-b349-a00139d7e220"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:00.893042 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.892952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/029756ea-96dc-4a43-b349-a00139d7e220-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "029756ea-96dc-4a43-b349-a00139d7e220" (UID: "029756ea-96dc-4a43-b349-a00139d7e220"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:59:00.893042 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.892976    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029756ea-96dc-4a43-b349-a00139d7e220-kube-api-access-dn5vr" (OuterVolumeSpecName: "kube-api-access-dn5vr") pod "029756ea-96dc-4a43-b349-a00139d7e220" (UID: "029756ea-96dc-4a43-b349-a00139d7e220"). InnerVolumeSpecName "kube-api-access-dn5vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:59:00.992695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.992647    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:00.992695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.992689    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/029756ea-96dc-4a43-b349-a00139d7e220-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:00.992695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.992700    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn5vr\" (UniqueName: \"kubernetes.io/projected/029756ea-96dc-4a43-b349-a00139d7e220-kube-api-access-dn5vr\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:00.992695 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:00.992708    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/029756ea-96dc-4a43-b349-a00139d7e220-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:01.788022 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:01.787993    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"
Jul 09 18:59:01.788022 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:01.788007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc" event={"ID":"029756ea-96dc-4a43-b349-a00139d7e220","Type":"ContainerDied","Data":"934b4d5fa465c684760772bb6f3b00e8bcef3d27278a730dcea6adba50836ab6"}
Jul 09 18:59:01.788560 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:01.788061    2581 scope.go:117] "RemoveContainer" containerID="eafaf761dcb1a18d9c4f86a0c645f39fcfdbbfe929e271eb0048281240e58ef5"
Jul 09 18:59:01.808913 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:01.808889    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"]
Jul 09 18:59:01.811084 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:01.811063    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4nwsqc"]
Jul 09 18:59:02.798904 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:02.798868    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="029756ea-96dc-4a43-b349-a00139d7e220" path="/var/lib/kubelet/pods/029756ea-96dc-4a43-b349-a00139d7e220/volumes"
Jul 09 18:59:07.286090 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.286056    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"]
Jul 09 18:59:07.286547 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.286532    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029756ea-96dc-4a43-b349-a00139d7e220" containerName="main"
Jul 09 18:59:07.286596 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.286549    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="029756ea-96dc-4a43-b349-a00139d7e220" containerName="main"
Jul 09 18:59:07.286640 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.286631    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="029756ea-96dc-4a43-b349-a00139d7e220" containerName="main"
Jul 09 18:59:07.290721 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.290702    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.295244 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.295220    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jul 09 18:59:07.295368 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.295279    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 18:59:07.295368 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.295329    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 18:59:07.295368 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.295344    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 18:59:07.304642 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.304617    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"]
Jul 09 18:59:07.326156 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.326130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:59:07.451703 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.451667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.451879 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.451724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.451879 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.451804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.451879 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.451845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8gg\" (UniqueName: \"kubernetes.io/projected/4ca84633-fb32-40de-a00b-4b1880b9db77-kube-api-access-9l8gg\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.451994 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.451879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca84633-fb32-40de-a00b-4b1880b9db77-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.451994 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.451903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.555958 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.555865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556108 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.555976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556108 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556215 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8gg\" (UniqueName: \"kubernetes.io/projected/4ca84633-fb32-40de-a00b-4b1880b9db77-kube-api-access-9l8gg\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556378 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca84633-fb32-40de-a00b-4b1880b9db77-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556750 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.556945 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.556818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.559418 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.559368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca84633-fb32-40de-a00b-4b1880b9db77-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.559648 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.559621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.565280 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.565176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8gg\" (UniqueName: \"kubernetes.io/projected/4ca84633-fb32-40de-a00b-4b1880b9db77-kube-api-access-9l8gg\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.586368 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.586338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:59:07.601355 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.601325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:07.736687 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.736662    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"]
Jul 09 18:59:07.739439 ip-10-0-143-79 kubenswrapper[2581]: W0709 18:59:07.739412    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca84633_fb32_40de_a00b_4b1880b9db77.slice/crio-d0f9f87df2a02ca5cb973699d6799e81f37e36471f172ba2fba725ce280621e2 WatchSource:0}: Error finding container d0f9f87df2a02ca5cb973699d6799e81f37e36471f172ba2fba725ce280621e2: Status 404 returned error can't find the container with id d0f9f87df2a02ca5cb973699d6799e81f37e36471f172ba2fba725ce280621e2
Jul 09 18:59:07.819212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.819110    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" event={"ID":"4ca84633-fb32-40de-a00b-4b1880b9db77","Type":"ContainerStarted","Data":"28d7af9eaefade8532b69561d67d887d922cc289e59fda6e08c22e6968b37b20"}
Jul 09 18:59:07.819212 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.819160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" event={"ID":"4ca84633-fb32-40de-a00b-4b1880b9db77","Type":"ContainerStarted","Data":"d0f9f87df2a02ca5cb973699d6799e81f37e36471f172ba2fba725ce280621e2"}
Jul 09 18:59:07.839672 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:07.839626    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" podStartSLOduration=0.839611508 podStartE2EDuration="839.611508ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:07.83717115 +0000 UTC m=+3341.760715619" watchObservedRunningTime="2026-07-09 18:59:07.839611508 +0000 UTC m=+3341.763156035"
Jul 09 18:59:08.601922 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:08.601883    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:14.549362 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:14.549328    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:59:17.601518 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:17.601481    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:17.613971 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:17.613939    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:17.881357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:17.881269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:40.826778 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:40.826741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:59:40.839887 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:40.839856    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"]
Jul 09 18:59:40.916235 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:40.916180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 18:59:55.862374 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:55.862310    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" podUID="4ca84633-fb32-40de-a00b-4b1880b9db77" containerName="main" containerID="cri-o://28d7af9eaefade8532b69561d67d887d922cc289e59fda6e08c22e6968b37b20" gracePeriod=45
Jul 09 18:59:55.873395 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:55.873309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" podUID="4ca84633-fb32-40de-a00b-4b1880b9db77" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Jul 09 18:59:56.060513 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.060474    2581 generic.go:356] "Generic (PLEG): container finished" podID="4ca84633-fb32-40de-a00b-4b1880b9db77" containerID="28d7af9eaefade8532b69561d67d887d922cc289e59fda6e08c22e6968b37b20" exitCode=0
Jul 09 18:59:56.060646 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.060539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" event={"ID":"4ca84633-fb32-40de-a00b-4b1880b9db77","Type":"ContainerDied","Data":"28d7af9eaefade8532b69561d67d887d922cc289e59fda6e08c22e6968b37b20"}
Jul 09 18:59:56.115884 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.115822    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:56.285701 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.285671    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-home\") pod \"4ca84633-fb32-40de-a00b-4b1880b9db77\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") "
Jul 09 18:59:56.285701 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.285708    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-tmp-dir\") pod \"4ca84633-fb32-40de-a00b-4b1880b9db77\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") "
Jul 09 18:59:56.285931 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.285730    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-model-cache\") pod \"4ca84633-fb32-40de-a00b-4b1880b9db77\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") "
Jul 09 18:59:56.285931 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.285750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-dshm\") pod \"4ca84633-fb32-40de-a00b-4b1880b9db77\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") "
Jul 09 18:59:56.285931 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.285778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l8gg\" (UniqueName: \"kubernetes.io/projected/4ca84633-fb32-40de-a00b-4b1880b9db77-kube-api-access-9l8gg\") pod \"4ca84633-fb32-40de-a00b-4b1880b9db77\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") "
Jul 09 18:59:56.285931 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.285883    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca84633-fb32-40de-a00b-4b1880b9db77-tls-certs\") pod \"4ca84633-fb32-40de-a00b-4b1880b9db77\" (UID: \"4ca84633-fb32-40de-a00b-4b1880b9db77\") "
Jul 09 18:59:56.286143 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.286002    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ca84633-fb32-40de-a00b-4b1880b9db77" (UID: "4ca84633-fb32-40de-a00b-4b1880b9db77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:56.286143 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.286012    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-home" (OuterVolumeSpecName: "home") pod "4ca84633-fb32-40de-a00b-4b1880b9db77" (UID: "4ca84633-fb32-40de-a00b-4b1880b9db77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:56.286143 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.286029    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ca84633-fb32-40de-a00b-4b1880b9db77" (UID: "4ca84633-fb32-40de-a00b-4b1880b9db77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:56.286314 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.286172    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:56.286314 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.286212    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:56.286314 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.286223    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:56.287919 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.287896    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ca84633-fb32-40de-a00b-4b1880b9db77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ca84633-fb32-40de-a00b-4b1880b9db77" (UID: "4ca84633-fb32-40de-a00b-4b1880b9db77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 18:59:56.288278 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.288253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-dshm" (OuterVolumeSpecName: "dshm") pod "4ca84633-fb32-40de-a00b-4b1880b9db77" (UID: "4ca84633-fb32-40de-a00b-4b1880b9db77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 18:59:56.288357 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.288271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca84633-fb32-40de-a00b-4b1880b9db77-kube-api-access-9l8gg" (OuterVolumeSpecName: "kube-api-access-9l8gg") pod "4ca84633-fb32-40de-a00b-4b1880b9db77" (UID: "4ca84633-fb32-40de-a00b-4b1880b9db77"). InnerVolumeSpecName "kube-api-access-9l8gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 18:59:56.387580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.387494    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ca84633-fb32-40de-a00b-4b1880b9db77-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:56.387580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.387525    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l8gg\" (UniqueName: \"kubernetes.io/projected/4ca84633-fb32-40de-a00b-4b1880b9db77-kube-api-access-9l8gg\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:56.387580 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:56.387535    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ca84633-fb32-40de-a00b-4b1880b9db77-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 18:59:57.066549 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:57.066480    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"
Jul 09 18:59:57.066959 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:57.066478    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c" event={"ID":"4ca84633-fb32-40de-a00b-4b1880b9db77","Type":"ContainerDied","Data":"d0f9f87df2a02ca5cb973699d6799e81f37e36471f172ba2fba725ce280621e2"}
Jul 09 18:59:57.066959 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:57.066610    2581 scope.go:117] "RemoveContainer" containerID="28d7af9eaefade8532b69561d67d887d922cc289e59fda6e08c22e6968b37b20"
Jul 09 18:59:57.084708 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:57.084686    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"]
Jul 09 18:59:57.086975 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:57.086955    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-wn96c"]
Jul 09 18:59:58.799586 ip-10-0-143-79 kubenswrapper[2581]: I0709 18:59:58.799550    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca84633-fb32-40de-a00b-4b1880b9db77" path="/var/lib/kubelet/pods/4ca84633-fb32-40de-a00b-4b1880b9db77/volumes"
Jul 09 19:00:07.689021 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.688986    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"]
Jul 09 19:00:07.689430 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.689415    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca84633-fb32-40de-a00b-4b1880b9db77" containerName="main"
Jul 09 19:00:07.689430 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.689431    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca84633-fb32-40de-a00b-4b1880b9db77" containerName="main"
Jul 09 19:00:07.689517 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.689508    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca84633-fb32-40de-a00b-4b1880b9db77" containerName="main"
Jul 09 19:00:07.694531 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.694501    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.696317 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.696289    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"]
Jul 09 19:00:07.697086 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.697068    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 19:00:07.698208 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.698038    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 19:00:07.698208 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.698091    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 19:00:07.698208 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.698134    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 09 19:00:07.890732 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.890699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-dshm\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.890732 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.890732    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-home\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.890921 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.890773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8lxc\" (UniqueName: \"kubernetes.io/projected/bba3487f-9eed-440b-a112-80ec8833ee0c-kube-api-access-g8lxc\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.890921 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.890833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-tmp-dir\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.890921 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.890884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba3487f-9eed-440b-a112-80ec8833ee0c-tls-certs\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.890921 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.890901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-model-cache\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.991814 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.991737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-dshm\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.991814 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.991772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-home\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.991895    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8lxc\" (UniqueName: \"kubernetes.io/projected/bba3487f-9eed-440b-a112-80ec8833ee0c-kube-api-access-g8lxc\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.991934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-tmp-dir\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.991996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba3487f-9eed-440b-a112-80ec8833ee0c-tls-certs\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.992041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-model-cache\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992332 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.992277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-home\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992332 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.992302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-tmp-dir\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.992414 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.992341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-model-cache\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.994052 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.994025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-dshm\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:07.994568 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:07.994550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba3487f-9eed-440b-a112-80ec8833ee0c-tls-certs\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:08.000458 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:08.000436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8lxc\" (UniqueName: \"kubernetes.io/projected/bba3487f-9eed-440b-a112-80ec8833ee0c-kube-api-access-g8lxc\") pod \"rolling-upgrade-test-kserve-54bc76f55b-5p2cd\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:08.006329 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:08.006305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:08.138982 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:08.138955    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"]
Jul 09 19:00:08.141704 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:00:08.141670    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba3487f_9eed_440b_a112_80ec8833ee0c.slice/crio-2167469f6b8587023c7f0cd1e654080eee4311b9c26b3323a9567e53701fe0f4 WatchSource:0}: Error finding container 2167469f6b8587023c7f0cd1e654080eee4311b9c26b3323a9567e53701fe0f4: Status 404 returned error can't find the container with id 2167469f6b8587023c7f0cd1e654080eee4311b9c26b3323a9567e53701fe0f4
Jul 09 19:00:08.523863 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:08.523824    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:00:08.620143 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:08.620113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:00:09.126320 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:09.126287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd" event={"ID":"bba3487f-9eed-440b-a112-80ec8833ee0c","Type":"ContainerStarted","Data":"1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3"}
Jul 09 19:00:09.126710 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:09.126337    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd" event={"ID":"bba3487f-9eed-440b-a112-80ec8833ee0c","Type":"ContainerStarted","Data":"2167469f6b8587023c7f0cd1e654080eee4311b9c26b3323a9567e53701fe0f4"}
Jul 09 19:00:09.148205 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:09.148134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd" podStartSLOduration=2.148115361 podStartE2EDuration="2.148115361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:09.146243153 +0000 UTC m=+3403.069787626" watchObservedRunningTime="2026-07-09 19:00:09.148115361 +0000 UTC m=+3403.071659836"
Jul 09 19:00:10.007165 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:10.007131    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:18.006746 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:18.006709    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:18.018880 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:18.018858    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:18.179998 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:18.179973    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:00:19.445650 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.445618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:00:19.705479 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.705407    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"]
Jul 09 19:00:19.708909 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.708884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.715591 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.715564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-x6rhh\""
Jul 09 19:00:19.715769 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.715564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 09 19:00:19.723248 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.723223    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"]
Jul 09 19:00:19.733496 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.733463    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:00:19.797294 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.797248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.797453 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.797368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.797453 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.797387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzkfw\" (UniqueName: \"kubernetes.io/projected/b784ab17-61ac-41a4-bdeb-5486ed753e3c-kube-api-access-tzkfw\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.897892 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.897857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.897892 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.897893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzkfw\" (UniqueName: \"kubernetes.io/projected/b784ab17-61ac-41a4-bdeb-5486ed753e3c-kube-api-access-tzkfw\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.898101 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.897946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.898377 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.898359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.900278 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.900258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:19.908222 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:19.908181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzkfw\" (UniqueName: \"kubernetes.io/projected/b784ab17-61ac-41a4-bdeb-5486ed753e3c-kube-api-access-tzkfw\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:20.021116 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:20.021033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:20.156445 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:20.156422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"]
Jul 09 19:00:20.162338 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:00:20.162308    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb784ab17_61ac_41a4_bdeb_5486ed753e3c.slice/crio-1ccd3f68c439dd99d765a0bf0d7d82dc220f65af73e471c52d27e69221d8bc84 WatchSource:0}: Error finding container 1ccd3f68c439dd99d765a0bf0d7d82dc220f65af73e471c52d27e69221d8bc84: Status 404 returned error can't find the container with id 1ccd3f68c439dd99d765a0bf0d7d82dc220f65af73e471c52d27e69221d8bc84
Jul 09 19:00:20.181671 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:20.181646    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" event={"ID":"b784ab17-61ac-41a4-bdeb-5486ed753e3c","Type":"ContainerStarted","Data":"1ccd3f68c439dd99d765a0bf0d7d82dc220f65af73e471c52d27e69221d8bc84"}
Jul 09 19:00:21.187983 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:21.187947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" event={"ID":"b784ab17-61ac-41a4-bdeb-5486ed753e3c","Type":"ContainerStarted","Data":"e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5"}
Jul 09 19:00:21.188371 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:21.188042    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:00:21.208667 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:21.208617    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podStartSLOduration=2.208602115 podStartE2EDuration="2.208602115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:21.20714023 +0000 UTC m=+3415.130684704" watchObservedRunningTime="2026-07-09 19:00:21.208602115 +0000 UTC m=+3415.132146586"
Jul 09 19:00:46.628339 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:46.628297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:00:52.195308 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:52.195268    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 09 19:00:59.344275 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:00:59.344170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:02.195786 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:02.195740    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 09 19:01:09.829213 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:09.829167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:12.195022 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.194983    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 09 19:01:12.226594 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.226552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:12.250622 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.250590    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"]
Jul 09 19:01:12.252248 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.252215    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 09 19:01:12.260253 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:01:12.260222    2581 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 09 19:01:12.260253 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 19:01:12.260253 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 19:01:12.260253 ip-10-0-143-79 kubenswrapper[2581]:  > containerID="e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5" cmd=["/bin/sleep","15"]
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.260272    2581 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" message="" err=<
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]:  >
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:01:12.260287    2581 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]:         , stdout: , stderr: , exit code -1
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]:  > pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main" containerID="cri-o://e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5"
Jul 09 19:01:12.260532 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.260327    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main" containerID="cri-o://e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5" gracePeriod=60
Jul 09 19:01:12.348229 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.348174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:12.453997 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.453931    2581 generic.go:356] "Generic (PLEG): container finished" podID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerID="e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5" exitCode=0
Jul 09 19:01:12.454127 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.454014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" event={"ID":"b784ab17-61ac-41a4-bdeb-5486ed753e3c","Type":"ContainerDied","Data":"e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5"}
Jul 09 19:01:12.516027 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.516005    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:01:12.570275 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.570241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tokenizer-uds\") pod \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") "
Jul 09 19:01:12.570275 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.570274    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzkfw\" (UniqueName: \"kubernetes.io/projected/b784ab17-61ac-41a4-bdeb-5486ed753e3c-kube-api-access-tzkfw\") pod \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") "
Jul 09 19:01:12.570483 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.570327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tls-certs\") pod \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\" (UID: \"b784ab17-61ac-41a4-bdeb-5486ed753e3c\") "
Jul 09 19:01:12.570547 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.570527    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b784ab17-61ac-41a4-bdeb-5486ed753e3c" (UID: "b784ab17-61ac-41a4-bdeb-5486ed753e3c"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:01:12.570683 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.570665    2581 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tokenizer-uds\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:12.572340 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.572310    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b784ab17-61ac-41a4-bdeb-5486ed753e3c-kube-api-access-tzkfw" (OuterVolumeSpecName: "kube-api-access-tzkfw") pod "b784ab17-61ac-41a4-bdeb-5486ed753e3c" (UID: "b784ab17-61ac-41a4-bdeb-5486ed753e3c"). InnerVolumeSpecName "kube-api-access-tzkfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:01:12.572435 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.572383    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b784ab17-61ac-41a4-bdeb-5486ed753e3c" (UID: "b784ab17-61ac-41a4-bdeb-5486ed753e3c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:01:12.672070 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.672035    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b784ab17-61ac-41a4-bdeb-5486ed753e3c-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:12.672070 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.672072    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzkfw\" (UniqueName: \"kubernetes.io/projected/b784ab17-61ac-41a4-bdeb-5486ed753e3c-kube-api-access-tzkfw\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:12.715467 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.715373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:12.716081 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:12.716060    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"]
Jul 09 19:01:13.256301 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:13.256261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:13.460146 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:13.460114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt" event={"ID":"b784ab17-61ac-41a4-bdeb-5486ed753e3c","Type":"ContainerDied","Data":"1ccd3f68c439dd99d765a0bf0d7d82dc220f65af73e471c52d27e69221d8bc84"}
Jul 09 19:01:13.460146 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:13.460137    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"
Jul 09 19:01:13.460396 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:13.460171    2581 scope.go:117] "RemoveContainer" containerID="e7c9e8555776c03bee354e57aa784c2b1c99e2cb8dd0e285ccffb252f31ef0f5"
Jul 09 19:01:13.478232 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:13.478206    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"]
Jul 09 19:01:13.482336 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:13.482315    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dvvddt"]
Jul 09 19:01:14.800608 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:14.800565    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" path="/var/lib/kubelet/pods/b784ab17-61ac-41a4-bdeb-5486ed753e3c/volumes"
Jul 09 19:01:27.736612 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:27.736550    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd" podUID="bba3487f-9eed-440b-a112-80ec8833ee0c" containerName="main" containerID="cri-o://1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3" gracePeriod=45
Jul 09 19:01:27.996392 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:27.996331    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:01:28.110761 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.110727    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-dshm\") pod \"bba3487f-9eed-440b-a112-80ec8833ee0c\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") "
Jul 09 19:01:28.110959 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.110776    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-home\") pod \"bba3487f-9eed-440b-a112-80ec8833ee0c\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") "
Jul 09 19:01:28.110959 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.110802    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-model-cache\") pod \"bba3487f-9eed-440b-a112-80ec8833ee0c\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") "
Jul 09 19:01:28.110959 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.110845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-tmp-dir\") pod \"bba3487f-9eed-440b-a112-80ec8833ee0c\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") "
Jul 09 19:01:28.110959 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.110889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8lxc\" (UniqueName: \"kubernetes.io/projected/bba3487f-9eed-440b-a112-80ec8833ee0c-kube-api-access-g8lxc\") pod \"bba3487f-9eed-440b-a112-80ec8833ee0c\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") "
Jul 09 19:01:28.110959 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.110924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba3487f-9eed-440b-a112-80ec8833ee0c-tls-certs\") pod \"bba3487f-9eed-440b-a112-80ec8833ee0c\" (UID: \"bba3487f-9eed-440b-a112-80ec8833ee0c\") "
Jul 09 19:01:28.111225 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.111037    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-home" (OuterVolumeSpecName: "home") pod "bba3487f-9eed-440b-a112-80ec8833ee0c" (UID: "bba3487f-9eed-440b-a112-80ec8833ee0c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:01:28.111225 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.111069    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-model-cache" (OuterVolumeSpecName: "model-cache") pod "bba3487f-9eed-440b-a112-80ec8833ee0c" (UID: "bba3487f-9eed-440b-a112-80ec8833ee0c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:01:28.111225 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.111109    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bba3487f-9eed-440b-a112-80ec8833ee0c" (UID: "bba3487f-9eed-440b-a112-80ec8833ee0c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:01:28.111369 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.111336    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:28.111369 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.111354    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:28.111369 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.111368    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:28.112951 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.112926    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bba3487f-9eed-440b-a112-80ec8833ee0c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bba3487f-9eed-440b-a112-80ec8833ee0c" (UID: "bba3487f-9eed-440b-a112-80ec8833ee0c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:01:28.113058 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.112979    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-dshm" (OuterVolumeSpecName: "dshm") pod "bba3487f-9eed-440b-a112-80ec8833ee0c" (UID: "bba3487f-9eed-440b-a112-80ec8833ee0c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:01:28.113101 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.113046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba3487f-9eed-440b-a112-80ec8833ee0c-kube-api-access-g8lxc" (OuterVolumeSpecName: "kube-api-access-g8lxc") pod "bba3487f-9eed-440b-a112-80ec8833ee0c" (UID: "bba3487f-9eed-440b-a112-80ec8833ee0c"). InnerVolumeSpecName "kube-api-access-g8lxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:01:28.212716 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.212681    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8lxc\" (UniqueName: \"kubernetes.io/projected/bba3487f-9eed-440b-a112-80ec8833ee0c-kube-api-access-g8lxc\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:28.212716 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.212714    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba3487f-9eed-440b-a112-80ec8833ee0c-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:28.212900 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.212730    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba3487f-9eed-440b-a112-80ec8833ee0c-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:01:28.533200 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.533144    2581 generic.go:356] "Generic (PLEG): container finished" podID="bba3487f-9eed-440b-a112-80ec8833ee0c" containerID="1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3" exitCode=0
Jul 09 19:01:28.533362 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.533216    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"
Jul 09 19:01:28.533362 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.533225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd" event={"ID":"bba3487f-9eed-440b-a112-80ec8833ee0c","Type":"ContainerDied","Data":"1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3"}
Jul 09 19:01:28.533362 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.533278    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd" event={"ID":"bba3487f-9eed-440b-a112-80ec8833ee0c","Type":"ContainerDied","Data":"2167469f6b8587023c7f0cd1e654080eee4311b9c26b3323a9567e53701fe0f4"}
Jul 09 19:01:28.533362 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.533299    2581 scope.go:117] "RemoveContainer" containerID="1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3"
Jul 09 19:01:28.542282 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.542262    2581 scope.go:117] "RemoveContainer" containerID="1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3"
Jul 09 19:01:28.542552 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:01:28.542534    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3\": container with ID starting with 1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3 not found: ID does not exist" containerID="1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3"
Jul 09 19:01:28.542620 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.542561    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3"} err="failed to get container status \"1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3\": rpc error: code = NotFound desc = could not find container \"1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3\": container with ID starting with 1b507151ca32b665dead5756a5e5ea31e381ad09998150b553bc229751562be3 not found: ID does not exist"
Jul 09 19:01:28.557140 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.557116    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"]
Jul 09 19:01:28.567102 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.567079    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-5p2cd"]
Jul 09 19:01:28.800144 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:28.800062    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bba3487f-9eed-440b-a112-80ec8833ee0c" path="/var/lib/kubelet/pods/bba3487f-9eed-440b-a112-80ec8833ee0c/volumes"
Jul 09 19:01:41.188151 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188118    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"]
Jul 09 19:01:41.188624 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188550    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bba3487f-9eed-440b-a112-80ec8833ee0c" containerName="main"
Jul 09 19:01:41.188624 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188562    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bba3487f-9eed-440b-a112-80ec8833ee0c" containerName="main"
Jul 09 19:01:41.188624 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188577    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main"
Jul 09 19:01:41.188624 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188583    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main"
Jul 09 19:01:41.188764 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188654    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bba3487f-9eed-440b-a112-80ec8833ee0c" containerName="main"
Jul 09 19:01:41.188764 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.188669    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b784ab17-61ac-41a4-bdeb-5486ed753e3c" containerName="main"
Jul 09 19:01:41.191597 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.191576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.194019 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.194002    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 19:01:41.195077 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.195060    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Jul 09 19:01:41.195137 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.195071    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 19:01:41.195137 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.195071    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 19:01:41.203034 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.203006    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"]
Jul 09 19:01:41.215710 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.215683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:41.227764 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.227724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-home\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.227764 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.227769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqzcg\" (UniqueName: \"kubernetes.io/projected/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-kube-api-access-lqzcg\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.227970 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.227809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tmp-dir\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.227970 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.227898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tls-certs\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.227970 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.227948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-dshm\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.227970 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.227969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-model-cache\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329397 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-model-cache\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329634 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-home\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329634 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqzcg\" (UniqueName: \"kubernetes.io/projected/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-kube-api-access-lqzcg\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329634 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tmp-dir\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329634 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tls-certs\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329858 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-dshm\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329858 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-model-cache\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.329858 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-home\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.330011 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.329962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tmp-dir\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.331821 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.331797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-dshm\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.332118 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.332100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tls-certs\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.342084 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.342061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqzcg\" (UniqueName: \"kubernetes.io/projected/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-kube-api-access-lqzcg\") pod \"scheduler-custom-template-test-kserve-5bc7bd6864-tj285\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.502174 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.502072    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:41.641059 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:41.640947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"]
Jul 09 19:01:41.643260 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:01:41.643234    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b287bee_ddbc_4089_860a_a5e1a9ae4a48.slice/crio-82817e24b1fd055a03ac0ab4e367f216bfab78b4b7c7bf409c3521e6bea37a12 WatchSource:0}: Error finding container 82817e24b1fd055a03ac0ab4e367f216bfab78b4b7c7bf409c3521e6bea37a12: Status 404 returned error can't find the container with id 82817e24b1fd055a03ac0ab4e367f216bfab78b4b7c7bf409c3521e6bea37a12
Jul 09 19:01:42.609395 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:42.609359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285" event={"ID":"1b287bee-ddbc-4089-860a-a5e1a9ae4a48","Type":"ContainerStarted","Data":"26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63"}
Jul 09 19:01:42.609764 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:42.609409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285" event={"ID":"1b287bee-ddbc-4089-860a-a5e1a9ae4a48","Type":"ContainerStarted","Data":"82817e24b1fd055a03ac0ab4e367f216bfab78b4b7c7bf409c3521e6bea37a12"}
Jul 09 19:01:42.630269 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:42.630166    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285" podStartSLOduration=1.6301465739999998 podStartE2EDuration="1.630146574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:01:42.627799457 +0000 UTC m=+3496.551343929" watchObservedRunningTime="2026-07-09 19:01:42.630146574 +0000 UTC m=+3496.553691047"
Jul 09 19:01:42.674985 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:42.674948    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:01:43.502899 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:43.502867    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:51.502738 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:51.502650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:51.515580 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:51.515552    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:01:51.664714 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:01:51.664686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:02:01.674610 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:01.674577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:02:29.810238 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:29.810205    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"]
Jul 09 19:02:29.818086 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:29.818055    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:02:30.202350 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:30.202311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:02:44.833206 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:44.833128    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285" podUID="1b287bee-ddbc-4089-860a-a5e1a9ae4a48" containerName="main" containerID="cri-o://26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63" gracePeriod=45
Jul 09 19:02:45.090528 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.090469    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:02:45.191923 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.191894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tls-certs\") pod \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") "
Jul 09 19:02:45.192117 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.191934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqzcg\" (UniqueName: \"kubernetes.io/projected/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-kube-api-access-lqzcg\") pod \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") "
Jul 09 19:02:45.192117 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.191958    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-model-cache\") pod \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") "
Jul 09 19:02:45.192117 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.191998    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-home\") pod \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") "
Jul 09 19:02:45.192117 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192056    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tmp-dir\") pod \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") "
Jul 09 19:02:45.192117 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192074    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-dshm\") pod \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\" (UID: \"1b287bee-ddbc-4089-860a-a5e1a9ae4a48\") "
Jul 09 19:02:45.192421 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192319    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-home" (OuterVolumeSpecName: "home") pod "1b287bee-ddbc-4089-860a-a5e1a9ae4a48" (UID: "1b287bee-ddbc-4089-860a-a5e1a9ae4a48"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:02:45.192421 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b287bee-ddbc-4089-860a-a5e1a9ae4a48" (UID: "1b287bee-ddbc-4089-860a-a5e1a9ae4a48"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:02:45.192421 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b287bee-ddbc-4089-860a-a5e1a9ae4a48" (UID: "1b287bee-ddbc-4089-860a-a5e1a9ae4a48"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:02:45.192558 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192423    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:02:45.192558 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192445    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:02:45.192558 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.192459    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:02:45.194075 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.194052    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-dshm" (OuterVolumeSpecName: "dshm") pod "1b287bee-ddbc-4089-860a-a5e1a9ae4a48" (UID: "1b287bee-ddbc-4089-860a-a5e1a9ae4a48"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:02:45.194154 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.194129    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b287bee-ddbc-4089-860a-a5e1a9ae4a48" (UID: "1b287bee-ddbc-4089-860a-a5e1a9ae4a48"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:02:45.194244 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.194230    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-kube-api-access-lqzcg" (OuterVolumeSpecName: "kube-api-access-lqzcg") pod "1b287bee-ddbc-4089-860a-a5e1a9ae4a48" (UID: "1b287bee-ddbc-4089-860a-a5e1a9ae4a48"). InnerVolumeSpecName "kube-api-access-lqzcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:02:45.293852 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.293801    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:02:45.293852 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.293845    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqzcg\" (UniqueName: \"kubernetes.io/projected/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-kube-api-access-lqzcg\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:02:45.293852 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.293856    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b287bee-ddbc-4089-860a-a5e1a9ae4a48-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:02:45.919138 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.919106    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b287bee-ddbc-4089-860a-a5e1a9ae4a48" containerID="26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63" exitCode=0
Jul 09 19:02:45.919575 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.919151    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"
Jul 09 19:02:45.919575 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.919153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285" event={"ID":"1b287bee-ddbc-4089-860a-a5e1a9ae4a48","Type":"ContainerDied","Data":"26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63"}
Jul 09 19:02:45.919575 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.919252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285" event={"ID":"1b287bee-ddbc-4089-860a-a5e1a9ae4a48","Type":"ContainerDied","Data":"82817e24b1fd055a03ac0ab4e367f216bfab78b4b7c7bf409c3521e6bea37a12"}
Jul 09 19:02:45.919575 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.919280    2581 scope.go:117] "RemoveContainer" containerID="26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63"
Jul 09 19:02:45.928484 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.928468    2581 scope.go:117] "RemoveContainer" containerID="26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63"
Jul 09 19:02:45.928796 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:02:45.928776    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63\": container with ID starting with 26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63 not found: ID does not exist" containerID="26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63"
Jul 09 19:02:45.928861 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.928805    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63"} err="failed to get container status \"26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63\": rpc error: code = NotFound desc = could not find container \"26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63\": container with ID starting with 26fcce7fdbc3e43edcf18913c934eadbe63670f79078a4127d64e419dd8ade63 not found: ID does not exist"
Jul 09 19:02:45.941676 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.941652    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"]
Jul 09 19:02:45.952865 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:45.952841    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-tj285"]
Jul 09 19:02:46.115367 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.115338    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"]
Jul 09 19:02:46.115780 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.115759    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b287bee-ddbc-4089-860a-a5e1a9ae4a48" containerName="main"
Jul 09 19:02:46.115780 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.115779    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b287bee-ddbc-4089-860a-a5e1a9ae4a48" containerName="main"
Jul 09 19:02:46.115976 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.115863    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b287bee-ddbc-4089-860a-a5e1a9ae4a48" containerName="main"
Jul 09 19:02:46.120287 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.120267    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.123457 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.123278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 19:02:46.123457 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.123311    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 09 19:02:46.123457 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.123340    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 19:02:46.123457 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.123367    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-dockercfg-rrjk7\""
Jul 09 19:02:46.123457 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.123419    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-w96lf\""
Jul 09 19:02:46.132799 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.132775    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"]
Jul 09 19:02:46.173681 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.173602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:02:46.200839 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.200813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.200971 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.200845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.200971 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.200862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.200971 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.200951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-home\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.201075 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.200974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckgtj\" (UniqueName: \"kubernetes.io/projected/d0302427-49f2-4204-8e7b-e42c33f53e89-kube-api-access-ckgtj\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.201075 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.201054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0302427-49f2-4204-8e7b-e42c33f53e89-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302503 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302503 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302705 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302705 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-home\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302705 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckgtj\" (UniqueName: \"kubernetes.io/projected/d0302427-49f2-4204-8e7b-e42c33f53e89-kube-api-access-ckgtj\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302835 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302754    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0302427-49f2-4204-8e7b-e42c33f53e89-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.302928 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.302903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.303110 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.303088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.303225 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.303160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-home\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.304825 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.304806    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.305144 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.305128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0302427-49f2-4204-8e7b-e42c33f53e89-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.315716 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.310820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckgtj\" (UniqueName: \"kubernetes.io/projected/d0302427-49f2-4204-8e7b-e42c33f53e89-kube-api-access-ckgtj\") pod \"scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.387682 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.387644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:02:46.432620 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.432531    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:46.574978 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.574952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"]
Jul 09 19:02:46.577450 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:02:46.577418    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0302427_49f2_4204_8e7b_e42c33f53e89.slice/crio-8c5d21687cf6583c3fc4d44ebd893b8c35188b256b2c0451de4639662fd2fc2c WatchSource:0}: Error finding container 8c5d21687cf6583c3fc4d44ebd893b8c35188b256b2c0451de4639662fd2fc2c: Status 404 returned error can't find the container with id 8c5d21687cf6583c3fc4d44ebd893b8c35188b256b2c0451de4639662fd2fc2c
Jul 09 19:02:46.579622 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.579607    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:02:46.801526 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.801444    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b287bee-ddbc-4089-860a-a5e1a9ae4a48" path="/var/lib/kubelet/pods/1b287bee-ddbc-4089-860a-a5e1a9ae4a48/volumes"
Jul 09 19:02:46.927058 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:46.927026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" event={"ID":"d0302427-49f2-4204-8e7b-e42c33f53e89","Type":"ContainerStarted","Data":"8c5d21687cf6583c3fc4d44ebd893b8c35188b256b2c0451de4639662fd2fc2c"}
Jul 09 19:02:47.933640 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:47.933604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" event={"ID":"d0302427-49f2-4204-8e7b-e42c33f53e89","Type":"ContainerStarted","Data":"e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721"}
Jul 09 19:02:47.934034 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:47.933678    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:48.940573 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:48.940536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" event={"ID":"d0302427-49f2-4204-8e7b-e42c33f53e89","Type":"ContainerStarted","Data":"626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568"}
Jul 09 19:02:48.964124 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:48.964071    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" podStartSLOduration=2.147056557 podStartE2EDuration="2.964054518s" totalImagesPullingTime="816.997961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:02:48.962407712 +0000 UTC m=+3562.885952183" watchObservedRunningTime="2026-07-09 19:02:48.964054518 +0000 UTC m=+3562.887598990"
Jul 09 19:02:49.433339 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:49.433306    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:54.590608 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:54.590571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:02:56.433692 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:56.433657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:56.445841 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:56.445817    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:56.988540 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:56.988515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:02:57.997303 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:02:57.997276    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:03:12.383766 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.383729    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:12.731411 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.731320    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"]
Jul 09 19:03:12.734853 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.734829    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.739120 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.739099    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Jul 09 19:03:12.739286 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.739269    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-z4jw9\""
Jul 09 19:03:12.747078 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.747055    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"]
Jul 09 19:03:12.771327 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.771295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:12.852461 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.852421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3ca42d-5a94-4458-b798-1441859d857c-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.852655 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.852470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3ca42d-5a94-4458-b798-1441859d857c-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.852655 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.852536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qmng\" (UniqueName: \"kubernetes.io/projected/5d3ca42d-5a94-4458-b798-1441859d857c-kube-api-access-7qmng\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.953891 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.953853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3ca42d-5a94-4458-b798-1441859d857c-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.953891 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.953894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3ca42d-5a94-4458-b798-1441859d857c-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.954098 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.953930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qmng\" (UniqueName: \"kubernetes.io/projected/5d3ca42d-5a94-4458-b798-1441859d857c-kube-api-access-7qmng\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.954322 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.954298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d3ca42d-5a94-4458-b798-1441859d857c-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.956422 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.956406    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d3ca42d-5a94-4458-b798-1441859d857c-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:12.962715 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:12.962686    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qmng\" (UniqueName: \"kubernetes.io/projected/5d3ca42d-5a94-4458-b798-1441859d857c-kube-api-access-7qmng\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth\" (UID: \"5d3ca42d-5a94-4458-b798-1441859d857c\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:13.043921 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:13.043843    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:13.181092 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:13.181067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"]
Jul 09 19:03:13.183250 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:03:13.183220    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3ca42d_5a94_4458_b798_1441859d857c.slice/crio-05b44a9f597e74525226cb2d4ac94f6ccc1fefcc76f2072d625309bdbfcfdaf3 WatchSource:0}: Error finding container 05b44a9f597e74525226cb2d4ac94f6ccc1fefcc76f2072d625309bdbfcfdaf3: Status 404 returned error can't find the container with id 05b44a9f597e74525226cb2d4ac94f6ccc1fefcc76f2072d625309bdbfcfdaf3
Jul 09 19:03:14.070136 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:14.070099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth" event={"ID":"5d3ca42d-5a94-4458-b798-1441859d857c","Type":"ContainerStarted","Data":"a3cab9d2bd17a2391fcf2e3f13b72af3473bc39be00ea51e41c4f3f48eb0041e"}
Jul 09 19:03:14.070538 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:14.070149    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth" event={"ID":"5d3ca42d-5a94-4458-b798-1441859d857c","Type":"ContainerStarted","Data":"05b44a9f597e74525226cb2d4ac94f6ccc1fefcc76f2072d625309bdbfcfdaf3"}
Jul 09 19:03:14.070538 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:14.070246    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:14.092368 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:14.092316    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth" podStartSLOduration=2.092298648 podStartE2EDuration="2.092298648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:14.089257339 +0000 UTC m=+3588.012801812" watchObservedRunningTime="2026-07-09 19:03:14.092298648 +0000 UTC m=+3588.015843120"
Jul 09 19:03:26.862529 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:26.862502    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:03:26.872758 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:26.872740    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:03:29.609028 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:29.608996    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:34.086981 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:34.086942    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"]
Jul 09 19:03:34.102805 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:34.102762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:34.196987 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:34.196951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:45.076982 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:45.076948    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth"
Jul 09 19:03:48.473715 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:48.473676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:48.575102 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:48.575069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:03:49.108364 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.108308    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="main" containerID="cri-o://626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568" gracePeriod=45
Jul 09 19:03:49.225134 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.225099    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="llm-d-routing-sidecar" containerID="cri-o://e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721" gracePeriod=45
Jul 09 19:03:49.242044 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.242013    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerID="626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568" exitCode=0
Jul 09 19:03:49.242179 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.242085    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" event={"ID":"d0302427-49f2-4204-8e7b-e42c33f53e89","Type":"ContainerDied","Data":"626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568"}
Jul 09 19:03:49.490920 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.490892    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:03:49.589676 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-home\") pod \"d0302427-49f2-4204-8e7b-e42c33f53e89\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") "
Jul 09 19:03:49.589838 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589703    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0302427-49f2-4204-8e7b-e42c33f53e89-tls-certs\") pod \"d0302427-49f2-4204-8e7b-e42c33f53e89\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") "
Jul 09 19:03:49.589838 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589740    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-model-cache\") pod \"d0302427-49f2-4204-8e7b-e42c33f53e89\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") "
Jul 09 19:03:49.589838 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-dshm\") pod \"d0302427-49f2-4204-8e7b-e42c33f53e89\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") "
Jul 09 19:03:49.589838 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckgtj\" (UniqueName: \"kubernetes.io/projected/d0302427-49f2-4204-8e7b-e42c33f53e89-kube-api-access-ckgtj\") pod \"d0302427-49f2-4204-8e7b-e42c33f53e89\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") "
Jul 09 19:03:49.589838 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589821    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-tmp-dir\") pod \"d0302427-49f2-4204-8e7b-e42c33f53e89\" (UID: \"d0302427-49f2-4204-8e7b-e42c33f53e89\") "
Jul 09 19:03:49.590057 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.589927    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-home" (OuterVolumeSpecName: "home") pod "d0302427-49f2-4204-8e7b-e42c33f53e89" (UID: "d0302427-49f2-4204-8e7b-e42c33f53e89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:49.590057 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.590027    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0302427-49f2-4204-8e7b-e42c33f53e89" (UID: "d0302427-49f2-4204-8e7b-e42c33f53e89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:49.590140 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.590117    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0302427-49f2-4204-8e7b-e42c33f53e89" (UID: "d0302427-49f2-4204-8e7b-e42c33f53e89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:49.590284 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.590267    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-home\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:03:49.590363 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.590286    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-model-cache\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:03:49.590363 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.590315    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-tmp-dir\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:03:49.591978 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.591948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0302427-49f2-4204-8e7b-e42c33f53e89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0302427-49f2-4204-8e7b-e42c33f53e89" (UID: "d0302427-49f2-4204-8e7b-e42c33f53e89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:03:49.592087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.591987    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-dshm" (OuterVolumeSpecName: "dshm") pod "d0302427-49f2-4204-8e7b-e42c33f53e89" (UID: "d0302427-49f2-4204-8e7b-e42c33f53e89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:49.592087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.591994    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0302427-49f2-4204-8e7b-e42c33f53e89-kube-api-access-ckgtj" (OuterVolumeSpecName: "kube-api-access-ckgtj") pod "d0302427-49f2-4204-8e7b-e42c33f53e89" (UID: "d0302427-49f2-4204-8e7b-e42c33f53e89"). InnerVolumeSpecName "kube-api-access-ckgtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:03:49.691604 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.691528    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0302427-49f2-4204-8e7b-e42c33f53e89-dshm\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:03:49.691604 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.691553    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckgtj\" (UniqueName: \"kubernetes.io/projected/d0302427-49f2-4204-8e7b-e42c33f53e89-kube-api-access-ckgtj\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:03:49.691604 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:49.691564    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0302427-49f2-4204-8e7b-e42c33f53e89-tls-certs\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:03:50.248443 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.248404    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerID="e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721" exitCode=0
Jul 09 19:03:50.248609 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.248480    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"
Jul 09 19:03:50.248609 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.248491    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" event={"ID":"d0302427-49f2-4204-8e7b-e42c33f53e89","Type":"ContainerDied","Data":"e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721"}
Jul 09 19:03:50.248609 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.248570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k" event={"ID":"d0302427-49f2-4204-8e7b-e42c33f53e89","Type":"ContainerDied","Data":"8c5d21687cf6583c3fc4d44ebd893b8c35188b256b2c0451de4639662fd2fc2c"}
Jul 09 19:03:50.248609 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.248592    2581 scope.go:117] "RemoveContainer" containerID="626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568"
Jul 09 19:03:50.257500 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.257482    2581 scope.go:117] "RemoveContainer" containerID="e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721"
Jul 09 19:03:50.265329 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.265312    2581 scope.go:117] "RemoveContainer" containerID="626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568"
Jul 09 19:03:50.265574 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:03:50.265555    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568\": container with ID starting with 626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568 not found: ID does not exist" containerID="626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568"
Jul 09 19:03:50.265641 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.265584    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568"} err="failed to get container status \"626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568\": rpc error: code = NotFound desc = could not find container \"626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568\": container with ID starting with 626d2f982f5f49bc40e6df2e8450018ec96595876f2e93c274d422ec8747b568 not found: ID does not exist"
Jul 09 19:03:50.265641 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.265606    2581 scope.go:117] "RemoveContainer" containerID="e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721"
Jul 09 19:03:50.265862 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:03:50.265843    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721\": container with ID starting with e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721 not found: ID does not exist" containerID="e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721"
Jul 09 19:03:50.265913 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.265867    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721"} err="failed to get container status \"e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721\": rpc error: code = NotFound desc = could not find container \"e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721\": container with ID starting with e65cfd5554045ca9b03378822b1147ab0039414b8cf21569de0a4d8dc4a89721 not found: ID does not exist"
Jul 09 19:03:50.279759 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.279737    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"]
Jul 09 19:03:50.294304 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.294283    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-8rx5k"]
Jul 09 19:03:50.800334 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:03:50.800302    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" path="/var/lib/kubelet/pods/d0302427-49f2-4204-8e7b-e42c33f53e89/volumes"
Jul 09 19:04:52.792599 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:04:52.792514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:05:31.997964 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:05:31.997926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:08:26.895235 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:08:26.895140    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:08:26.904165 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:08:26.904147    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:08:40.233053 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:08:40.233012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:09:10.606901 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:09:10.606860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:12:22.304274 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:12:22.304176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:12:33.657780 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:12:33.657749    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:13:26.924577 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:13:26.924470    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:13:26.934368 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:13:26.934346    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:14:05.068962 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:14:05.068891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:17:01.934203 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:17:01.934113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:17:01.991409 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:17:01.991374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:17:23.662292 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:17:23.662257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:18:26.955477 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:18:26.955366    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:18:26.966469 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:18:26.966449    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:18:35.529346 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:18:35.529309    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:18:35.631824 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:18:35.631789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:19:15.262659 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:19:15.262630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:22:10.652707 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:10.652661    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:14.247787 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.247748    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jst5r/must-gather-rkht6"]
Jul 09 19:22:14.248269 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.248253    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="llm-d-routing-sidecar"
Jul 09 19:22:14.248326 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.248270    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="llm-d-routing-sidecar"
Jul 09 19:22:14.248326 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.248290    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="main"
Jul 09 19:22:14.248326 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.248295    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="main"
Jul 09 19:22:14.248422 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.248361    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="llm-d-routing-sidecar"
Jul 09 19:22:14.248422 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.248369    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0302427-49f2-4204-8e7b-e42c33f53e89" containerName="main"
Jul 09 19:22:14.251478 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.251463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.254152 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.254126    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jst5r\"/\"openshift-service-ca.crt\""
Jul 09 19:22:14.254312 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.254213    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jst5r\"/\"default-dockercfg-l9587\""
Jul 09 19:22:14.254312 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.254217    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jst5r\"/\"kube-root-ca.crt\""
Jul 09 19:22:14.273624 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.273596    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jst5r/must-gather-rkht6"]
Jul 09 19:22:14.285990 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.285967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:22:14.364739 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.364688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:22:14.385903 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.385875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv84q\" (UniqueName: \"kubernetes.io/projected/d946ddbf-a58e-4d2c-acd2-00114f866310-kube-api-access-sv84q\") pod \"must-gather-rkht6\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") " pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.386033 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.385950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d946ddbf-a58e-4d2c-acd2-00114f866310-must-gather-output\") pod \"must-gather-rkht6\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") " pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.486547 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.486512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv84q\" (UniqueName: \"kubernetes.io/projected/d946ddbf-a58e-4d2c-acd2-00114f866310-kube-api-access-sv84q\") pod \"must-gather-rkht6\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") " pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.486692 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.486608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d946ddbf-a58e-4d2c-acd2-00114f866310-must-gather-output\") pod \"must-gather-rkht6\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") " pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.486905 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.486889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d946ddbf-a58e-4d2c-acd2-00114f866310-must-gather-output\") pod \"must-gather-rkht6\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") " pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.495012 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.494981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv84q\" (UniqueName: \"kubernetes.io/projected/d946ddbf-a58e-4d2c-acd2-00114f866310-kube-api-access-sv84q\") pod \"must-gather-rkht6\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") " pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.560765 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.560692    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:22:14.706725 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.706702    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jst5r/must-gather-rkht6"]
Jul 09 19:22:14.708879 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:22:14.708849    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd946ddbf_a58e_4d2c_acd2_00114f866310.slice/crio-8ed740fba3686a65541cbc8b146425f7767911f30089ccb34d0292dd50ed3c86 WatchSource:0}: Error finding container 8ed740fba3686a65541cbc8b146425f7767911f30089ccb34d0292dd50ed3c86: Status 404 returned error can't find the container with id 8ed740fba3686a65541cbc8b146425f7767911f30089ccb34d0292dd50ed3c86
Jul 09 19:22:14.710717 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:14.710700    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:22:15.595848 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:15.595780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jst5r/must-gather-rkht6" event={"ID":"d946ddbf-a58e-4d2c-acd2-00114f866310","Type":"ContainerStarted","Data":"8ed740fba3686a65541cbc8b146425f7767911f30089ccb34d0292dd50ed3c86"}
Jul 09 19:22:21.634375 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:21.634320    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jst5r/must-gather-rkht6" event={"ID":"d946ddbf-a58e-4d2c-acd2-00114f866310","Type":"ContainerStarted","Data":"59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97"}
Jul 09 19:22:21.634375 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:21.634386    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jst5r/must-gather-rkht6" event={"ID":"d946ddbf-a58e-4d2c-acd2-00114f866310","Type":"ContainerStarted","Data":"fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"}
Jul 09 19:22:21.653810 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:21.653754    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jst5r/must-gather-rkht6" podStartSLOduration=1.360897266 podStartE2EDuration="7.653732938s" totalImagesPullingTime="6.292835672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:22:21.651866542 +0000 UTC m=+4735.575411015" watchObservedRunningTime="2026-07-09 19:22:21.653732938 +0000 UTC m=+4735.577277412"
Jul 09 19:22:30.619547 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:30.619511    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:32.224047 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:32.224016    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:33.845368 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:33.845336    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:35.459860 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:35.459821    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:37.063854 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:37.063824    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:38.718654 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:38.718620    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:40.315129 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:40.315096    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:42.064378 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:42.064340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:43.795801 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:43.795774    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:45.409684 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:45.409651    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:47.036088 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:47.036053    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:48.746940 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:48.746863    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:50.481424 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:50.481394    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:52.081795 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:52.081755    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:53.726354 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:53.726325    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:22:55.462008 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:22:55.461978    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_scheduler-v06-pd-migration-test-kserve-router-scheduler-597gpth_5d3ca42d-5a94-4458-b798-1441859d857c/main/0.log"
Jul 09 19:23:02.101825 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:02.101743    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:02.118151 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:02.118123    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:02.133785 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:02.133764    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:02.345665 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:02.345634    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:03.469498 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:03.469469    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:03.484150 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:03.484120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:03.498615 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:03.498593    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:03.696526 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:03.696497    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:04.811747 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:04.811721    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:04.826937 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:04.826914    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:04.841018 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:04.840997    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:05.046088 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:05.046056    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:06.119244 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:06.119213    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:06.133584 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:06.133561    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:06.151340 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:06.151318    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:06.369509 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:06.369443    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:07.471080 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:07.471053    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:07.485773 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:07.485750    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:07.500161 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:07.500135    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:07.711302 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:07.711270    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:08.814645 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:08.814619    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:08.832809 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:08.832789    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:08.848798 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:08.848777    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:09.062720 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:09.062693    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:10.159638 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:10.159610    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:10.175471 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:10.175448    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:10.191574 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:10.191547    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:10.396804 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:10.396779    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:11.528493 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:11.528462    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:11.548149 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:11.548124    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:11.571533 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:11.571509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:11.801764 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:11.801696    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:13.003563 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:13.003533    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:13.018582 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:13.018558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:13.033484 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:13.033463    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:13.243159 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:13.243135    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:14.343026 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:14.342992    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:14.358536 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:14.358510    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:14.373915 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:14.373893    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:14.591498 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:14.591471    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:15.672942 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:15.672910    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:15.687030 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:15.687010    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:15.701359 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:15.701336    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:15.907170 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:15.907143    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:16.947907 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:16.947873    2581 generic.go:356] "Generic (PLEG): container finished" podID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerID="fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc" exitCode=0
Jul 09 19:23:16.948345 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:16.947945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jst5r/must-gather-rkht6" event={"ID":"d946ddbf-a58e-4d2c-acd2-00114f866310","Type":"ContainerDied","Data":"fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"}
Jul 09 19:23:16.948345 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:16.948308    2581 scope.go:117] "RemoveContainer" containerID="fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"
Jul 09 19:23:17.735413 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:17.735383    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-jst5r_must-gather-rkht6_d946ddbf-a58e-4d2c-acd2-00114f866310/gather/0.log"
Jul 09 19:23:18.398106 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:18.398072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:23:18.574399 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:18.574369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:23:18.695414 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:18.695330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:23:21.548235 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:21.548203    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7c7ll_c1c19302-1cda-47c9-94aa-ac799575abe2/global-pull-secret-syncer/0.log"
Jul 09 19:23:21.679386 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:21.679354    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log"
Jul 09 19:23:21.850815 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:21.850794    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-79.ec2.internal_28e83951a50f7c287e9df4e6d546594a/haproxy/0.log"
Jul 09 19:23:23.378524 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.378488    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-jst5r/must-gather-rkht6"]
Jul 09 19:23:23.378994 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.378702    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-jst5r/must-gather-rkht6" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="copy" containerID="cri-o://59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97" gracePeriod=2
Jul 09 19:23:23.380601 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.380576    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-jst5r/must-gather-rkht6"]
Jul 09 19:23:23.380838 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:23:23.380808    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rkht6\" is forbidden: User \"system:node:ip-10-0-143-79.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-jst5r\": no relationship found between node 'ip-10-0-143-79.ec2.internal' and this object" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:23:23.631755 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.631691    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-jst5r_must-gather-rkht6_d946ddbf-a58e-4d2c-acd2-00114f866310/copy/0.log"
Jul 09 19:23:23.634269 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.634248    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:23:23.636387 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:23:23.636359    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rkht6\" is forbidden: User \"system:node:ip-10-0-143-79.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-jst5r\": no relationship found between node 'ip-10-0-143-79.ec2.internal' and this object" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:23:23.737081 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.737046    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv84q\" (UniqueName: \"kubernetes.io/projected/d946ddbf-a58e-4d2c-acd2-00114f866310-kube-api-access-sv84q\") pod \"d946ddbf-a58e-4d2c-acd2-00114f866310\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") "
Jul 09 19:23:23.737274 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.737163    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d946ddbf-a58e-4d2c-acd2-00114f866310-must-gather-output\") pod \"d946ddbf-a58e-4d2c-acd2-00114f866310\" (UID: \"d946ddbf-a58e-4d2c-acd2-00114f866310\") "
Jul 09 19:23:23.739587 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.739559    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d946ddbf-a58e-4d2c-acd2-00114f866310-kube-api-access-sv84q" (OuterVolumeSpecName: "kube-api-access-sv84q") pod "d946ddbf-a58e-4d2c-acd2-00114f866310" (UID: "d946ddbf-a58e-4d2c-acd2-00114f866310"). InnerVolumeSpecName "kube-api-access-sv84q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:23:23.744295 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.744271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d946ddbf-a58e-4d2c-acd2-00114f866310-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "d946ddbf-a58e-4d2c-acd2-00114f866310" (UID: "d946ddbf-a58e-4d2c-acd2-00114f866310"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:23:23.838495 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.838465    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv84q\" (UniqueName: \"kubernetes.io/projected/d946ddbf-a58e-4d2c-acd2-00114f866310-kube-api-access-sv84q\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:23:23.838495 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.838490    2581 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d946ddbf-a58e-4d2c-acd2-00114f866310-must-gather-output\") on node \"ip-10-0-143-79.ec2.internal\" DevicePath \"\""
Jul 09 19:23:23.983728 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.983648    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-jst5r_must-gather-rkht6_d946ddbf-a58e-4d2c-acd2-00114f866310/copy/0.log"
Jul 09 19:23:23.984046 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.984020    2581 generic.go:356] "Generic (PLEG): container finished" podID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerID="59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97" exitCode=143
Jul 09 19:23:23.984208 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.984067    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:23:23.984208 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.984133    2581 scope.go:117] "RemoveContainer" containerID="59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97"
Jul 09 19:23:23.986175 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:23:23.986146    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rkht6\" is forbidden: User \"system:node:ip-10-0-143-79.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-jst5r\": no relationship found between node 'ip-10-0-143-79.ec2.internal' and this object" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:23:23.992943 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:23.992924    2581 scope.go:117] "RemoveContainer" containerID="fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"
Jul 09 19:23:23.994760 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:23:23.994736    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-rkht6\" is forbidden: User \"system:node:ip-10-0-143-79.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-jst5r\": no relationship found between node 'ip-10-0-143-79.ec2.internal' and this object" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" pod="openshift-must-gather-jst5r/must-gather-rkht6"
Jul 09 19:23:24.005359 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.005337    2581 scope.go:117] "RemoveContainer" containerID="59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97"
Jul 09 19:23:24.005639 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:23:24.005620    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97\": container with ID starting with 59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97 not found: ID does not exist" containerID="59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97"
Jul 09 19:23:24.005689 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.005647    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97"} err="failed to get container status \"59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97\": rpc error: code = NotFound desc = could not find container \"59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97\": container with ID starting with 59d7dc4022d5c271403ff185e39bfdfc04f1d8a63cc06cfa92a5f5ce88f01a97 not found: ID does not exist"
Jul 09 19:23:24.005689 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.005665    2581 scope.go:117] "RemoveContainer" containerID="fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"
Jul 09 19:23:24.005889 ip-10-0-143-79 kubenswrapper[2581]: E0709 19:23:24.005872    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc\": container with ID starting with fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc not found: ID does not exist" containerID="fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"
Jul 09 19:23:24.005942 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.005900    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc"} err="failed to get container status \"fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc\": rpc error: code = NotFound desc = could not find container \"fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc\": container with ID starting with fa8ab59c2b47300da246282e8699c50de8c18a8a025b5efc9079dc4ae7b6d2fc not found: ID does not exist"
Jul 09 19:23:24.223377 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.223348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:23:24.345590 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.345505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44zx7_b54996e8-faeb-4a8a-bc84-9ab7e37276db/konnectivity-agent/0.log" limit=1024
Jul 09 19:23:24.800451 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:24.800418    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" path="/var/lib/kubelet/pods/d946ddbf-a58e-4d2c-acd2-00114f866310/volumes"
Jul 09 19:23:26.504175 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:26.504142    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-trkjt_14223905-4d68-4886-9f9f-e1dff711bac0/manager/0.log"
Jul 09 19:23:26.532289 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:26.532260    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-94pfn_5672237d-5983-4c0c-874c-10bcf5c09cad/manager/0.log"
Jul 09 19:23:26.565065 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:26.565041    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-r9kdf_deb31e7c-4910-4fb8-ab8f-a2a61cc0560b/kuadrant-console-plugin/0.log"
Jul 09 19:23:26.809174 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:26.809058    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-xw8v8_d449c6ba-4429-4bed-a25d-09af63bc817b/limitador/0.log"
Jul 09 19:23:26.985994 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:26.985887    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:23:27.005788 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:26.998469    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hvbkx_ffdf63a8-f00b-4da3-b68c-e45a92e1af50/ovn-acl-logging/0.log"
Jul 09 19:23:28.238533 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.238504    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/alertmanager/0.log"
Jul 09 19:23:28.265806 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.265781    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/config-reloader/0.log"
Jul 09 19:23:28.292676 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.292645    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/kube-rbac-proxy-web/0.log"
Jul 09 19:23:28.317066 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.317043    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/kube-rbac-proxy/0.log"
Jul 09 19:23:28.341637 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.341611    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/kube-rbac-proxy-metric/0.log"
Jul 09 19:23:28.368832 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.368808    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/prom-label-proxy/0.log"
Jul 09 19:23:28.393075 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.393049    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7fbe37bc-a838-4e1b-8749-935e68bef66a/init-config-reloader/0.log"
Jul 09 19:23:28.613525 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.613497    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-cmr54_3255187c-5222-4fbb-89a7-0507b85ea06e/kube-state-metrics/0.log"
Jul 09 19:23:28.636171 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.636141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-cmr54_3255187c-5222-4fbb-89a7-0507b85ea06e/kube-rbac-proxy-main/0.log"
Jul 09 19:23:28.662078 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.662043    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-cmr54_3255187c-5222-4fbb-89a7-0507b85ea06e/kube-rbac-proxy-self/0.log"
Jul 09 19:23:28.690973 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.690949    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fb696657c-dww2m_235c39b9-9046-4a1d-abbb-210284c39933/metrics-server/0.log"
Jul 09 19:23:28.724759 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.724732    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-plj97_8868369b-4bd8-4d79-abcb-4ac42db6fa0f/monitoring-plugin/0.log"
Jul 09 19:23:28.874987 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.874906    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8tnlx_2e2874ca-13c1-432b-b37c-2ebe4fc1fc03/node-exporter/0.log"
Jul 09 19:23:28.917121 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.917098    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8tnlx_2e2874ca-13c1-432b-b37c-2ebe4fc1fc03/kube-rbac-proxy/0.log"
Jul 09 19:23:28.942076 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:28.942050    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8tnlx_2e2874ca-13c1-432b-b37c-2ebe4fc1fc03/init-textfile/0.log"
Jul 09 19:23:29.080812 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.080784    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kpdm5_6fb29d9d-a69b-4ba4-92a5-dacad1c693aa/kube-rbac-proxy-main/0.log"
Jul 09 19:23:29.108948 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.108920    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kpdm5_6fb29d9d-a69b-4ba4-92a5-dacad1c693aa/kube-rbac-proxy-self/0.log"
Jul 09 19:23:29.132693 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.132622    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kpdm5_6fb29d9d-a69b-4ba4-92a5-dacad1c693aa/openshift-state-metrics/0.log"
Jul 09 19:23:29.184635 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.184607    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/prometheus/0.log"
Jul 09 19:23:29.203038 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.203017    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/config-reloader/0.log"
Jul 09 19:23:29.230375 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.230352    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/thanos-sidecar/0.log"
Jul 09 19:23:29.255753 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.255729    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/kube-rbac-proxy-web/0.log"
Jul 09 19:23:29.285093 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.285071    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/kube-rbac-proxy/0.log"
Jul 09 19:23:29.309365 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.309340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/kube-rbac-proxy-thanos/0.log"
Jul 09 19:23:29.335820 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.335801    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82eb4f29-899a-427b-aff2-98ee9062506f/init-config-reloader/0.log"
Jul 09 19:23:29.369861 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.369838    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-kmfk8_c85b356f-d322-47de-aaef-946709cb5e5a/prometheus-operator/0.log"
Jul 09 19:23:29.392600 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.391698    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-kmfk8_c85b356f-d322-47de-aaef-946709cb5e5a/kube-rbac-proxy/0.log"
Jul 09 19:23:29.418968 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.418937    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-fnqrs_099445ac-a094-476d-94c2-a109fb2a053e/prometheus-operator-admission-webhook/0.log"
Jul 09 19:23:29.453526 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.453501    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7695ff9586-d576f_3f0d55cd-cf73-445b-a342-41cf3c46fe3d/telemeter-client/0.log"
Jul 09 19:23:29.477858 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.477833    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7695ff9586-d576f_3f0d55cd-cf73-445b-a342-41cf3c46fe3d/reload/0.log"
Jul 09 19:23:29.506088 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.506060    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7695ff9586-d576f_3f0d55cd-cf73-445b-a342-41cf3c46fe3d/kube-rbac-proxy/0.log"
Jul 09 19:23:29.541659 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.541636    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5558858-grx86_a31a3680-a10f-43bc-a288-54b6bc6f8035/thanos-query/0.log"
Jul 09 19:23:29.564777 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.564755    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5558858-grx86_a31a3680-a10f-43bc-a288-54b6bc6f8035/kube-rbac-proxy-web/0.log"
Jul 09 19:23:29.588200 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.588168    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5558858-grx86_a31a3680-a10f-43bc-a288-54b6bc6f8035/kube-rbac-proxy/0.log"
Jul 09 19:23:29.613087 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.613054    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5558858-grx86_a31a3680-a10f-43bc-a288-54b6bc6f8035/prom-label-proxy/0.log"
Jul 09 19:23:29.637424 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.637400    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5558858-grx86_a31a3680-a10f-43bc-a288-54b6bc6f8035/kube-rbac-proxy-rules/0.log"
Jul 09 19:23:29.660657 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:29.660584    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7cb5558858-grx86_a31a3680-a10f-43bc-a288-54b6bc6f8035/kube-rbac-proxy-metrics/0.log"
Jul 09 19:23:31.352104 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352069    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"]
Jul 09 19:23:31.352517 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352500    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="gather"
Jul 09 19:23:31.352566 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352519    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="gather"
Jul 09 19:23:31.352566 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352535    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="copy"
Jul 09 19:23:31.352566 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352540    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="copy"
Jul 09 19:23:31.352658 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352610    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="copy"
Jul 09 19:23:31.352658 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.352621    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d946ddbf-a58e-4d2c-acd2-00114f866310" containerName="gather"
Jul 09 19:23:31.356908 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.356893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.359099 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.359077    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vv9mp\"/\"default-dockercfg-rrxh9\""
Jul 09 19:23:31.359286 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.359263    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vv9mp\"/\"openshift-service-ca.crt\""
Jul 09 19:23:31.360093 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.360079    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vv9mp\"/\"kube-root-ca.crt\""
Jul 09 19:23:31.363444 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.363421    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"]
Jul 09 19:23:31.411139 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.411112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-proc\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.411304 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.411153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkdrj\" (UniqueName: \"kubernetes.io/projected/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-kube-api-access-bkdrj\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.411304 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.411242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-podres\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.411304 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.411267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-sys\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.411304 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.411288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-lib-modules\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512240 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-sys\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512240 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-lib-modules\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512447 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-proc\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512447 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-sys\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512447 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkdrj\" (UniqueName: \"kubernetes.io/projected/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-kube-api-access-bkdrj\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512447 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-proc\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512447 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512420    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-lib-modules\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512447 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-podres\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.512643 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.512497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-podres\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.520303 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.520281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkdrj\" (UniqueName: \"kubernetes.io/projected/b39dd79f-f0bd-4892-bea8-dae523b5a0c4-kube-api-access-bkdrj\") pod \"perf-node-gather-daemonset-mz2p6\" (UID: \"b39dd79f-f0bd-4892-bea8-dae523b5a0c4\") " pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.668497 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.668415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:31.801304 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.801198    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"]
Jul 09 19:23:31.804086 ip-10-0-143-79 kubenswrapper[2581]: W0709 19:23:31.804056    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb39dd79f_f0bd_4892_bea8_dae523b5a0c4.slice/crio-ee216af0168d382d2d282ffad8fc6f5a904e4c7195b285521540fd179ac738ab WatchSource:0}: Error finding container ee216af0168d382d2d282ffad8fc6f5a904e4c7195b285521540fd179ac738ab: Status 404 returned error can't find the container with id ee216af0168d382d2d282ffad8fc6f5a904e4c7195b285521540fd179ac738ab
Jul 09 19:23:31.878110 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:31.878084    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-f54r7_6d16fd67-63e1-4a66-8088-5f8c62bd72cb/download-server/0.log"
Jul 09 19:23:32.024514 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:32.024467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6" event={"ID":"b39dd79f-f0bd-4892-bea8-dae523b5a0c4","Type":"ContainerStarted","Data":"1dfbfc9453c708a8185e07259099fc7e0dfac707e15a4d12283aa2403442bf45"}
Jul 09 19:23:32.024732 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:32.024533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6" event={"ID":"b39dd79f-f0bd-4892-bea8-dae523b5a0c4","Type":"ContainerStarted","Data":"ee216af0168d382d2d282ffad8fc6f5a904e4c7195b285521540fd179ac738ab"}
Jul 09 19:23:32.024732 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:32.024650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:32.043507 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:32.043446    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6" podStartSLOduration=1.043427495 podStartE2EDuration="1.043427495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:23:32.041498878 +0000 UTC m=+4805.965043352" watchObservedRunningTime="2026-07-09 19:23:32.043427495 +0000 UTC m=+4805.966971969"
Jul 09 19:23:33.506616 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:33.506578    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gc54s_83404659-7147-40f3-a6e4-7fb8a477d296/dns/0.log"
Jul 09 19:23:33.530005 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:33.529980    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gc54s_83404659-7147-40f3-a6e4-7fb8a477d296/kube-rbac-proxy/0.log"
Jul 09 19:23:33.637251 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:33.637225    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8kwh5_022963e4-a05c-488f-b0c2-b8dbd79305eb/dns-node-resolver/0.log"
Jul 09 19:23:34.337826 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:34.337801    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wn8fk_6297deef-388a-44dc-8125-5a565307e957/node-ca/0.log"
Jul 09 19:23:36.252244 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:36.252218    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4srdt_5ee0ea18-4568-47f0-a889-8f1e623f51ac/serve-healthcheck-canary/0.log"
Jul 09 19:23:36.793831 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:36.793789    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f2fxq_7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5/kube-rbac-proxy/0.log"
Jul 09 19:23:36.817214 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:36.817174    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f2fxq_7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5/exporter/0.log"
Jul 09 19:23:36.840901 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:36.840879    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f2fxq_7c1d1a14-13fc-4fa7-a6dd-dbbedd9b18e5/extractor/0.log"
Jul 09 19:23:38.039157 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:38.039131    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vv9mp/perf-node-gather-daemonset-mz2p6"
Jul 09 19:23:40.093510 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:40.093483    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7885457d66-b8k5x_5c11113e-3839-4563-ab30-b90de9b2247e/manager/0.log"
Jul 09 19:23:41.774287 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:41.774258    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-lrtg7_3ecd8a36-969e-4f19-837b-e4c164de1b50/seaweedfs/0.log"
Jul 09 19:23:48.363387 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.363354    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8lfv8_452a82e7-5977-4a7e-9449-7fc238ecd9fd/kube-multus/0.log"
Jul 09 19:23:48.792619 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.792535    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/kube-multus-additional-cni-plugins/0.log"
Jul 09 19:23:48.819169 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.819140    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/egress-router-binary-copy/0.log"
Jul 09 19:23:48.843816 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.843786    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/cni-plugins/0.log"
Jul 09 19:23:48.870958 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.870925    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/bond-cni-plugin/0.log"
Jul 09 19:23:48.897300 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.897273    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/routeoverride-cni/0.log"
Jul 09 19:23:48.923982 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.923960    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/whereabouts-cni-bincopy/0.log"
Jul 09 19:23:48.950786 ip-10-0-143-79 kubenswrapper[2581]: I0709 19:23:48.950757    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jpjpd_1b9f6d09-bbda-4c8d-a4c7-39a062fa1681/whereabouts-cni/0.log"