Jul 08 06:54:31.016210 ip-10-0-129-54 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 06:54:31.465365 ip-10-0-129-54 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 06:54:31.465365 ip-10-0-129-54 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 06:54:31.465365 ip-10-0-129-54 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 06:54:31.465365 ip-10-0-129-54 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 06:54:31.465365 ip-10-0-129-54 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 06:54:31.466400 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.466319    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 06:54:31.471776 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471761    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:54:31.471776 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471777    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471781    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471784    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471788    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471791    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471794    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471797    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471800    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471802    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471805    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471808    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471812    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471816    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471819    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471827    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471830    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471833    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471835    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471838    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471840    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:54:31.471836 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471843    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471846    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471849    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471852    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471854    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471856    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471859    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471862    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471865    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471867    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471870    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471872    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471875    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471877    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471880    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471884    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471887    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471890    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471895    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:54:31.472344 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471898    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471900    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471903    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471906    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471908    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471910    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471913    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471915    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471918    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471920    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471923    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471925    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471927    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471930    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471933    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471936    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471938    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471941    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471957    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471960    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:54:31.472788 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471962    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471965    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471968    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471970    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471973    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471976    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471978    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471981    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471983    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471986    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471989    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471992    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471995    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.471998    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472000    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472002    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472005    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472007    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472010    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:54:31.473305 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472012    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472015    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472017    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472020    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472023    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472025    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472028    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472031    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472034    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472037    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472039    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472042    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472044    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472047    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472050    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472052    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472055    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472057    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472060    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:54:31.473974 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472064    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472737    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472745    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472749    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472752    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472756    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472759    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472762    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472764    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472767    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472769    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472772    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472775    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472777    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472780    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472782    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472785    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472787    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472790    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:54:31.474657 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472793    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472795    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472800    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472803    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472805    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472808    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472810    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472815    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472818    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472820    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472823    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472825    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472828    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472831    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472833    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472836    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472838    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472841    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472845    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472848    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:54:31.475151 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472851    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472854    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472872    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472877    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472886    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472891    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472895    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472898    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472900    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472903    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472906    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472908    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472911    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472914    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472917    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472920    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472922    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472925    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472927    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:54:31.475666 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472930    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472933    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472935    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472938    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472940    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472955    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472958    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472961    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472963    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472966    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472968    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472971    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472973    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472976    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472978    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472982    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472984    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472987    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472990    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:54:31.476137 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472993    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472995    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.472998    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473000    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473003    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473005    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473008    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473011    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473014    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473017    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473020    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473022    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473025    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473028    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473030    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473033    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473035    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473038    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473040    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473043    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:54:31.476604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473046    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473048    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.473051    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474568    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474583    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474590    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474595    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474599    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474604    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474608    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474612    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474616    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474620    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474625    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474629    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474633    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474637    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474641    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474646    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474650    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:54:31.477102 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474656    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474663    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474667    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474671    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474675    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474679    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474685    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474690    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474695    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474700    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474704    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474709    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474713    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474722    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474726    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474730    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474733    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474736    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474739    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474741    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:54:31.477609 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474744    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474747    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474750    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474753    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474756    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474758    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474761    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474764    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474767    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474769    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474771    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474776    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474779    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474781    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474784    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474786    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474789    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474792    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474794    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:54:31.478092 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474797    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474799    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474801    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474804    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474807    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474809    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474812    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474814    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474817    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474820    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474824    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474827    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474830    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474833    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474835    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474838    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474861    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474866    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474869    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474872    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:54:31.478545 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474874    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474877    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474880    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474882    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474889    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474892    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474895    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474898    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474901    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474903    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474905    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474908    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474911    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474913    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474916    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474918    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474921    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474923    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474926    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:54:31.479047 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474929    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474932    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474935    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.474937    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475028    2571 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475038    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475045    2571 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475052    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475057    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475060    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475064    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475068    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475071    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475074    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475077    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475082    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475086    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475091    2571 flags.go:64] FLAG: --cgroup-root=""
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475096    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475100    2571 flags.go:64] FLAG: --client-ca-file=""
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475104    2571 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475108    2571 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475112    2571 flags.go:64] FLAG: --cluster-domain=""
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475115    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 06:54:31.479509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475118    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475121    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475124    2571 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475128    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475130    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475133    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475136    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475139    2571 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475142    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475146    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475149    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475152    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475156    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475159    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475162    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475165    2571 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475168    2571 flags.go:64] FLAG: --enable-server="true"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475171    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475175    2571 flags.go:64] FLAG: --event-burst="100"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475178    2571 flags.go:64] FLAG: --event-qps="50"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475181    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475184    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475187    2571 flags.go:64] FLAG: --eviction-hard=""
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475195    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 06:54:31.480105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475198    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475201    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475204    2571 flags.go:64] FLAG: --eviction-soft=""
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475206    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475209    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475212    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475215    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475218    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475221    2571 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475224    2571 flags.go:64] FLAG: --feature-gates=""
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475227    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475230    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475233    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475236    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475239    2571 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475241    2571 flags.go:64] FLAG: --help="false"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475244    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475248    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475250    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475253    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475256    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475260    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475263    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 06:54:31.480664 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475265    2571 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475268    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475271    2571 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475274    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475277    2571 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475280    2571 flags.go:64] FLAG: --kube-reserved=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475282    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475285    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475287    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475290    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475293    2571 flags.go:64] FLAG: --lock-file=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475295    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475298    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475301    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475306    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475309    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475312    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475314    2571 flags.go:64] FLAG: --logging-format="text"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475317    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475320    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475323    2571 flags.go:64] FLAG: --manifest-url=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475326    2571 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475329    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475332    2571 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475336    2571 flags.go:64] FLAG: --max-pods="110"
Jul 08 06:54:31.481298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475339    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475342    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475345    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475347    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475350    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475353    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475356    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475363    2571 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475366    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475369    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475372    2571 flags.go:64] FLAG: --pod-cidr=""
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475374    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475380    2571 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475382    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475385    2571 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475388    2571 flags.go:64] FLAG: --port="10250"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475391    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475394    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-079f6e1b8a85f366a"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475396    2571 flags.go:64] FLAG: --qos-reserved=""
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475399    2571 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475402    2571 flags.go:64] FLAG: --register-node="true"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475405    2571 flags.go:64] FLAG: --register-with-taints=""
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475409    2571 flags.go:64] FLAG: --registry-burst="10"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475411    2571 flags.go:64] FLAG: --registry-qps="5"
Jul 08 06:54:31.481890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475414    2571 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475417    2571 flags.go:64] FLAG: --reserved-memory=""
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475425    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475428    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475431    2571 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475434    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475437    2571 flags.go:64] FLAG: --runonce="false"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475439    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475445    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475448    2571 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475450    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475453    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475456    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475459    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475462    2571 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475464    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475467    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475470    2571 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475472    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475475    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475478    2571 flags.go:64] FLAG: --system-cgroups=""
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475481    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475486    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475488    2571 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475491    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 06:54:31.482508 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475495    2571 flags.go:64] FLAG: --tls-min-version=""
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475498    2571 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475501    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475503    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475506    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475509    2571 flags.go:64] FLAG: --v="2"
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475513    2571 flags.go:64] FLAG: --version="false"
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475516    2571 flags.go:64] FLAG: --vmodule=""
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475520    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.475524    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475675    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475679    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475682    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475685    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475688    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475694    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475699    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475702    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475705    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475709    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475712    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475714    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:54:31.483154 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475717    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475720    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475724    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475728    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475731    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475734    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475736    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475739    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475742    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475744    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475746    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475749    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475752    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475755    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475757    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475760    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475762    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475765    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475768    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:54:31.483676 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475771    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475773    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475776    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475778    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475781    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475784    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475788    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475791    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475793    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475796    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475799    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475802    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475806    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475808    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475811    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475814    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475817    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475820    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475822    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475825    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:54:31.484152 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475828    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475831    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475833    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475836    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475838    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475841    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475844    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475846    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475848    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475851    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475854    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475856    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475858    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475861    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475864    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475866    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475869    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475875    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475879    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:54:31.484655 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475882    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475884    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475887    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475890    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475892    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475895    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475898    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475901    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475903    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475906    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475908    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475911    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475914    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475916    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475918    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475921    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475923    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475926    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475928    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475931    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:54:31.485207 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475933    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475935    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475938    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475940    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475960    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475965    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475968    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475971    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.475973    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.476632    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.483816    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.483831    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.483850    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.483855    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483915    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:54:31.485674 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483920    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483924    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483927    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483931    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483934    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483936    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483939    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483942    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483967    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483970    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483975    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483979    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483982    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483986    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483989    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483992    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483995    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.483997    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484000    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:54:31.486070 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484003    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484005    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484008    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484011    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484014    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484017    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484020    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484023    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484025    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484029    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484033    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484037    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484040    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484042    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484045    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484048    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484051    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484053    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484056    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:54:31.486536 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484058    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484061    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484064    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484066    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484069    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484071    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484074    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484076    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484079    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484081    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484084    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484086    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484089    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484091    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484094    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484096    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484099    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484102    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484104    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484108    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:54:31.487030 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484110    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484113    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484116    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484119    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484121    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484124    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484127    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484129    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484132    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484134    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484137    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484139    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484142    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484144    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484147    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484149    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484152    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484154    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484158    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484161    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:54:31.487527 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484163    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484166    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484169    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484171    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484174    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484176    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484179    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484181    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484184    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484186    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484189    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484191    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484195    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484199    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484203    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484206    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484209    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484212    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484215    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:54:31.488040 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484217    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.484222    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484335    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484339    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484341    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484344    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484346    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484349    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484351    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484354    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484356    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484359    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484361    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484364    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484366    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 06:54:31.488493 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484369    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484371    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484374    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484376    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484378    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484381    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484383    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484386    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484388    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484391    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484393    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484396    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484399    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484402    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484404    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484406    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484409    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484412    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484414    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484417    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 06:54:31.488859 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484419    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484422    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484424    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484427    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484429    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484432    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484435    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484437    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484440    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484442    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484445    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484447    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484450    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484452    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484454    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484457    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484459    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484462    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484464    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484467    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 06:54:31.489343 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484469    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484472    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484474    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484477    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484480    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484484    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484487    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484490    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484493    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484497    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484500    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484503    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484506    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484508    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484511    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484514    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484516    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484519    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484522    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 06:54:31.489829 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484524    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484527    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484529    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484532    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484534    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484537    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484539    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484541    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484544    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484547    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484549    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484552    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484554    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484557    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484559    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484562    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484564    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484567    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484571    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484573    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 06:54:31.490304 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484576    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484578    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484581    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484584    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484586    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484589    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:31.484592    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.484597    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.484704    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.487038    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.488573    2571 server.go:1019] "Starting client certificate rotation"
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.488665    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 06:54:31.490779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.488702    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 06:54:31.491370 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.491357    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 06:54:31.493371 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.493357    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 06:54:31.502913 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.502887    2571 log.go:25] "Validated CRI v1 runtime API"
Jul 08 06:54:31.507914 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.507895    2571 log.go:25] "Validated CRI v1 image API"
Jul 08 06:54:31.508766 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.508741    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 06:54:31.511249 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.511230    2571 fs.go:135] Filesystem UUIDs: map[1c9d1cfb-9166-4bb7-aa29-a8aff315abfb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c05a480f-7d5d-4523-aa3c-fd16c1bf586a:/dev/nvme0n1p4]
Jul 08 06:54:31.511323 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.511248    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 06:54:31.515542 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.515437    2571 manager.go:216] Machine: {Timestamp:2026-07-08 06:54:31.514972468 +0000 UTC m=+0.362000429 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115146 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22f49d03c3cc759f243ab7d7940203 SystemUUID:ec22f49d-03c3-cc75-9f24-3ab7d7940203 BootID:fbc1d875-0f2e-4a99-a720-9211e3607206 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:67:e1:77:8c:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:67:e1:77:8c:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:80:7e:f9:80:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 06:54:31.515542 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.515536    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 06:54:31.515640 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.515611    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 06:54:31.519392 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.519354    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 06:54:31.519598 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.519396    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 06:54:31.520366 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.520353    2571 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 06:54:31.520439 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.520370    2571 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 06:54:31.520439 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.520380    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 06:54:31.522831 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.522817    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 06:54:31.526089 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.526075    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 06:54:31.529614 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529599    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 06:54:31.529763 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529751    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 06:54:31.529830 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529801    2571 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 06:54:31.529830 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529816    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 06:54:31.529919 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529837    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 06:54:31.529919 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529850    2571 kubelet.go:398] "Adding apiserver pod source"
Jul 08 06:54:31.529919 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.529861    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 06:54:31.530274 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.530260    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 06:54:31.530391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.530284    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 06:54:31.532269 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.532250    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 06:54:31.534181 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.534166    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 06:54:31.535920 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.535906    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 06:54:31.536000 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.535936    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 06:54:31.538300 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538284    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 06:54:31.538357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538313    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 06:54:31.538357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538323    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 06:54:31.538357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538346    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538360    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538366    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538374    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538379    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538385    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538396    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538408    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 06:54:31.538443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.538414    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 06:54:31.539877 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.539864    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 06:54:31.539877 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.539875    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 06:54:31.541939 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.541915    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 06:54:31.541939 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.541914    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 06:54:31.542098 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.542048    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 06:54:31.543011 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.543000    2571 server.go:1268] "Started kubelet"
Jul 08 06:54:31.543129 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.543076    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 06:54:31.543181 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.543156    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 06:54:31.543764 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.543733    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 06:54:31.543906 ip-10-0-129-54 systemd[1]: Started Kubernetes Kubelet.
Jul 08 06:54:31.544892 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.544733    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 06:54:31.546603 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.546555    2571 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 06:54:31.549394 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.548383    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b7f79ebf0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.542975472 +0000 UTC m=+0.390003432,LastTimestamp:2026-07-08 06:54:31.542975472 +0000 UTC m=+0.390003432,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.549663 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.549646    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 06:54:31.551545 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.551529    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 06:54:31.551618 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.551550    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 06:54:31.551668 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.551638    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:31.552233 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552217    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 06:54:31.552319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552236    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 06:54:31.552319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552217    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 06:54:31.552319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552317    2571 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 06:54:31.552437 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552325    2571 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 06:54:31.552437 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552408    2571 factory.go:55] Registering systemd factory
Jul 08 06:54:31.552437 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552425    2571 factory.go:223] Registration of the systemd container factory successfully
Jul 08 06:54:31.552622 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552608    2571 factory.go:153] Registering CRI-O factory
Jul 08 06:54:31.552674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552625    2571 factory.go:223] Registration of the crio container factory successfully
Jul 08 06:54:31.552720 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552675    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 06:54:31.552720 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552699    2571 factory.go:103] Registering Raw factory
Jul 08 06:54:31.552720 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.552714    2571 manager.go:1196] Started watching for new ooms in manager
Jul 08 06:54:31.553185 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.553172    2571 manager.go:349] Starting recovery of all containers
Jul 08 06:54:31.563623 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.563607    2571 manager.go:354] Recovery completed
Jul 08 06:54:31.566155 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.566128    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 06:54:31.566356 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.566331    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 06:54:31.568125 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.568108    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.569797 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.569779    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.569863 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.569811    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.569863 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.569822    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.570356 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.570343    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.570424 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.570346    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 06:54:31.570424 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.570411    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 06:54:31.570507 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.570433    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 06:54:31.571029 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.571012    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.571106 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.571044    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.571106 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.571059    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.571558 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.571547    2571 policy_none.go:49] "None policy: Start"
Jul 08 06:54:31.571592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.571562    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 06:54:31.571592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.571572    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 06:54:31.572535 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.572459    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b81132f54  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-54.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.569796948 +0000 UTC m=+0.416824919,LastTimestamp:2026-07-08 06:54:31.569796948 +0000 UTC m=+0.416824919,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.572535 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.572531    2571 policy_none.go:47] "Start"
Jul 08 06:54:31.587168 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.587003    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b81137b56  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-54.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.569816406 +0000 UTC m=+0.416844368,LastTimestamp:2026-07-08 06:54:31.569816406 +0000 UTC m=+0.416844368,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.592514 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.592452    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b8113a065  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-54.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.569825893 +0000 UTC m=+0.416853854,LastTimestamp:2026-07-08 06:54:31.569825893 +0000 UTC m=+0.416853854,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.601540 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.601474    2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-54.ec2.internal.18c03e0b81132f54\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b81132f54  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-54.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.569796948 +0000 UTC m=+0.416824919,LastTimestamp:2026-07-08 06:54:31.571028113 +0000 UTC m=+0.418056077,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.606228 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.606144    2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-54.ec2.internal.18c03e0b81137b56\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b81137b56  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-54.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.569816406 +0000 UTC m=+0.416844368,LastTimestamp:2026-07-08 06:54:31.571050975 +0000 UTC m=+0.418078937,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.612210 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.612195    2571 manager.go:337] "Starting Device Plugin manager"
Jul 08 06:54:31.612313 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.612226    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 06:54:31.612313 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.612236    2571 server.go:87] "Starting device plugin registration server"
Jul 08 06:54:31.612313 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.612203    2571 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-54.ec2.internal.18c03e0b8113a065\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b8113a065  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-54.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.569825893 +0000 UTC m=+0.416853854,LastTimestamp:2026-07-08 06:54:31.571069273 +0000 UTC m=+0.418097233,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.612487 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.612475    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 06:54:31.612527 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.612491    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 06:54:31.613194 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.612703    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 06:54:31.613319 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.613214    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 06:54:31.613319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.613232    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 06:54:31.613319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.613243    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 06:54:31.613319 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.613266    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:31.623613 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.623537    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-54.ec2.internal.18c03e0b83bc64ae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-54.ec2.internal,UID:ip-10-0-129-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-54.ec2.internal,},FirstTimestamp:2026-07-08 06:54:31.614440622 +0000 UTC m=+0.461468569,LastTimestamp:2026-07-08 06:54:31.614440622 +0000 UTC m=+0.461468569,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-54.ec2.internal,}"
Jul 08 06:54:31.636746 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.636728    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bd472"
Jul 08 06:54:31.645140 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.645124    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bd472"
Jul 08 06:54:31.658187 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.658153    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 06:54:31.659466 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.659447    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 06:54:31.659549 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.659489    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 06:54:31.659549 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.659508    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 06:54:31.659549 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.659543    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 06:54:31.662070 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.662048    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 08 06:54:31.713562 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.713534    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.714403 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.714387    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.714450 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.714419    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.714450 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.714430    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.714509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.714452    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.737553 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.737531    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.737646 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.737555    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-54.ec2.internal\": node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:31.760177 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.760153    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"]
Jul 08 06:54:31.760240 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.760209    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.761009 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.760993    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.761087 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.761024    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.761087 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.761036    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.761493 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.761476    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:31.762190 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762178    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.762305 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762289    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.762341 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762323    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.762927 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762903    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.763042 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762930    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.763042 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762986    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.763042 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.763003    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.763042 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.762932    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.763042 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.763040    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.766298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.766282    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.766356 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.766317    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 06:54:31.767019 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.767003    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 06:54:31.767093 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.767035    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 06:54:31.767093 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.767048    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 06:54:31.795509 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.795490    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-54.ec2.internal\" not found" node="ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.799792 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.799778    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-54.ec2.internal\" not found" node="ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.862815 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.862547    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:31.953421 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.953387    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e1ff6d8fbf84e1b6a71573c64b922f8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal\" (UID: \"3e1ff6d8fbf84e1b6a71573c64b922f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.953421 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.953420    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e1ff6d8fbf84e1b6a71573c64b922f8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal\" (UID: \"3e1ff6d8fbf84e1b6a71573c64b922f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.953600 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:31.953443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8ad56ce288e22a2bd7487c8f13b9062-config\") pod \"kube-apiserver-proxy-ip-10-0-129-54.ec2.internal\" (UID: \"c8ad56ce288e22a2bd7487c8f13b9062\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:31.963489 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:31.963469    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:32.054207 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.054144    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e1ff6d8fbf84e1b6a71573c64b922f8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal\" (UID: \"3e1ff6d8fbf84e1b6a71573c64b922f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.054207 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.054159    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e1ff6d8fbf84e1b6a71573c64b922f8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal\" (UID: \"3e1ff6d8fbf84e1b6a71573c64b922f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.054207 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.054187    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e1ff6d8fbf84e1b6a71573c64b922f8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal\" (UID: \"3e1ff6d8fbf84e1b6a71573c64b922f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.054207 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.054205    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8ad56ce288e22a2bd7487c8f13b9062-config\") pod \"kube-apiserver-proxy-ip-10-0-129-54.ec2.internal\" (UID: \"c8ad56ce288e22a2bd7487c8f13b9062\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.054367 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.054240    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8ad56ce288e22a2bd7487c8f13b9062-config\") pod \"kube-apiserver-proxy-ip-10-0-129-54.ec2.internal\" (UID: \"c8ad56ce288e22a2bd7487c8f13b9062\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.054367 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.054273    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e1ff6d8fbf84e1b6a71573c64b922f8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal\" (UID: \"3e1ff6d8fbf84e1b6a71573c64b922f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.064241 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.064215    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:32.098395 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.098364    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.103256 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.103242    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.164616 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.164586    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:32.265150 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.265120    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:32.365694 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.365625    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-54.ec2.internal\" not found"
Jul 08 06:54:32.402212 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.402193    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:54:32.421115 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.421095    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:54:32.451904 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.451881    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.464824 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.464807    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 06:54:32.466139 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.466124    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal"
Jul 08 06:54:32.475621 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.475602    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 06:54:32.487751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.487733    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 06:54:32.487898 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.487880    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 06:54:32.530410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.530389    2571 apiserver.go:52] "Watching apiserver"
Jul 08 06:54:32.540263 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.540243    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 06:54:32.540602 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.540582    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-lb9c2","openshift-network-operator/iptables-alerter-prtsv","kube-system/global-pull-secret-syncer-6qqpr","kube-system/konnectivity-agent-kd766","openshift-cluster-node-tuning-operator/tuned-gmw5z","openshift-dns/node-resolver-b8bd9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal","openshift-multus/network-metrics-daemon-vtskv","openshift-ovn-kubernetes/ovnkube-node-n5zzg","kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt","openshift-image-registry/node-ca-kkdj4","openshift-multus/multus-additional-cni-plugins-xt7ss","openshift-multus/multus-psd4s"]
Jul 08 06:54:32.542037 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.542020    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.543189 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.543175    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:32.543745 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.543467    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:32.544603 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.544585    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.544978 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.544940    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.545302 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.545280    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-snv62\""
Jul 08 06:54:32.545428 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.545398    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.546182 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.546165    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.547410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.547396    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.547467 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.547444    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:32.547859 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.547844    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 06:54:32.548054 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.548039    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hp5zz\""
Jul 08 06:54:32.548115 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.548101    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 06:54:32.548276 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.548260    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.548740 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.548729    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:32.548795 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.548781    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:32.549073 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.549056    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.549110 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.549087    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w95z2\""
Jul 08 06:54:32.549783 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.549771    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.550995 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.550979    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.551965 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.551927    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 06:54:32.552188 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.552159    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.552188 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.552180    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.552310 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.552282    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 06:54:32.552568 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.552552    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-76mjg\""
Jul 08 06:54:32.552861 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.552845    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.553017 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.552995    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 06:54:32.553137 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.553125    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 06:54:32.553290 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.553275    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.553393 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.553373    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 06:54:32.553611 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.553600    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.554490 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.554471    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.554908 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.554891    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4lg2s\""
Jul 08 06:54:32.555011 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.554929    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 06:54:32.555253 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.555238    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 06:54:32.555331 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.555290    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.555897 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.555881    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-twm4w\""
Jul 08 06:54:32.555997 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.555897    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556132    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556177    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysctl-d\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556234    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-systemd\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-cni-bin\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556279    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-env-overrides\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-kubelet-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-socket-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556346    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-registration-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556369    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovnkube-config\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-systemd\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-var-lib-kubelet\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556438    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/486b3034-61ca-49ad-9061-bf7c9369447c-kubelet-config\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556460    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-run-netns\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-cni-netd\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556506    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5de8057-31c1-410d-9f8e-f5479e1c644a-iptables-alerter-script\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.556577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-modprobe-d\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556586    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysctl-conf\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556612    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-run-ovn-kubernetes\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556700    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-device-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556796    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556861    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556924    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556968    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8v7ct\""
Jul 08 06:54:32.557410 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.556734    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-sys-fs\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557623    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-systemd-units\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557643    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-ovn\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557657    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovn-node-metrics-cert\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557670    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557672    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovnkube-script-lib\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557781    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1fd811b-ecd4-4720-b927-556e91ace214-tmp-dir\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.557833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-kubelet\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557837    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkth8\" (UniqueName: \"kubernetes.io/projected/e1fd811b-ecd4-4720-b927-556e91ace214-kube-api-access-xkth8\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557878    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ll9d\" (UniqueName: \"kubernetes.io/projected/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-kube-api-access-8ll9d\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-tmp\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.557929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/66b62ff9-9859-419a-9aa0-365b4f25229b-agent-certs\") pod \"konnectivity-agent-kd766\" (UID: \"66b62ff9-9859-419a-9aa0-365b4f25229b\") " pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/66b62ff9-9859-419a-9aa0-365b4f25229b-konnectivity-ca\") pod \"konnectivity-agent-kd766\" (UID: \"66b62ff9-9859-419a-9aa0-365b4f25229b\") " pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558033    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-slash\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpk7q\" (UniqueName: \"kubernetes.io/projected/01e967d7-7ae9-4bd1-a51c-74608439db60-kube-api-access-wpk7q\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1fd811b-ecd4-4720-b927-556e91ace214-hosts-file\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5de8057-31c1-410d-9f8e-f5479e1c644a-host-slash\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.558149 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558137    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-run\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558160    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysconfig\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558182    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-host\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcnng\" (UniqueName: \"kubernetes.io/projected/e5de8057-31c1-410d-9f8e-f5479e1c644a-kube-api-access-fcnng\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558254    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-etc-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-node-log\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558307    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-log-socket\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558329    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-kubernetes\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558354    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-sys\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558375    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/486b3034-61ca-49ad-9061-bf7c9369447c-dbus\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558393    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-var-lib-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558414    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-lib-modules\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558442    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86pcs\" (UniqueName: \"kubernetes.io/projected/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-kube-api-access-86pcs\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gvp2\" (UniqueName: \"kubernetes.io/projected/524b3e5f-fe27-42de-b73e-a0eea84945bf-kube-api-access-8gvp2\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.558518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558507    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-tuned\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.558995 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.558546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-etc-selinux\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.559255 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.559243    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 06:54:32.559309 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.559296    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 06:54:32.559473 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.559460    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c7jfv\""
Jul 08 06:54:32.559623 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.559609    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 06:54:32.559849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.559838    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hs7f5\""
Jul 08 06:54:32.560092 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.560079    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 06:54:32.560488 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.560476    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 06:54:32.562050 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.562035    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 06:54:32.567816 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.567797    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 06:54:32.592070 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.592052    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:54:32.600942 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.600927    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pjkhs"
Jul 08 06:54:32.609177 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.609161    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pjkhs"
Jul 08 06:54:32.646827 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.646789    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 06:49:32 +0000 UTC" deadline="2028-01-19 02:37:02.705982353 +0000 UTC"
Jul 08 06:54:32.646827 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.646827    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13435h42m30.059158098s"
Jul 08 06:54:32.652917 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.652903    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 06:54:32.659647 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659629    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-modprobe-d\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.659692 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659658    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.659692 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659675    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:32.659792 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659691    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysctl-conf\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.659792 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659742    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.659792 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.659770    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:32.659792 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659782    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-modprobe-d\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.659792 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659786    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-run-ovn-kubernetes\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.659836    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:33.159805044 +0000 UTC m=+2.006832993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-device-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-run-ovn-kubernetes\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659864    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-sys-fs\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659831    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysctl-conf\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.659868    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659903    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-device-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659915    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-sys-fs\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659879    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65032d2e-e6b0-4469-9921-3273ef4e5f75-host\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.659926    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:54:33.159908753 +0000 UTC m=+2.006936707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.659997    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-systemd-units\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-ovn\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660054    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-ovn\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovn-node-metrics-cert\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660086    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovnkube-script-lib\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660106    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1fd811b-ecd4-4720-b927-556e91ace214-tmp-dir\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660115    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-systemd-units\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660125    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-kubelet\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-kubelet\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660157    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkth8\" (UniqueName: \"kubernetes.io/projected/e1fd811b-ecd4-4720-b927-556e91ace214-kube-api-access-xkth8\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-os-release\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-netns\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660208    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-kubelet\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-cni-multus\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn4jb\" (UniqueName: \"kubernetes.io/projected/12276eee-8052-47e3-83b1-f1ba413727e2-kube-api-access-xn4jb\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ll9d\" (UniqueName: \"kubernetes.io/projected/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-kube-api-access-8ll9d\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660321    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-tmp\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660397    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1fd811b-ecd4-4720-b927-556e91ace214-tmp-dir\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660389    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660420    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/66b62ff9-9859-419a-9aa0-365b4f25229b-agent-certs\") pod \"konnectivity-agent-kd766\" (UID: \"66b62ff9-9859-419a-9aa0-365b4f25229b\") " pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660463    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/66b62ff9-9859-419a-9aa0-365b4f25229b-konnectivity-ca\") pod \"konnectivity-agent-kd766\" (UID: \"66b62ff9-9859-419a-9aa0-365b4f25229b\") " pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.660680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660619    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-slash\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660743    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-slash\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660765    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovnkube-script-lib\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660785    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpk7q\" (UniqueName: \"kubernetes.io/projected/01e967d7-7ae9-4bd1-a51c-74608439db60-kube-api-access-wpk7q\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660909    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1fd811b-ecd4-4720-b927-556e91ace214-hosts-file\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5de8057-31c1-410d-9f8e-f5479e1c644a-host-slash\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.660987    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-cnibin\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661006    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e1fd811b-ecd4-4720-b927-556e91ace214-hosts-file\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-etc-kubernetes\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661043    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5de8057-31c1-410d-9f8e-f5479e1c644a-host-slash\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-run\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661086    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-run\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661089    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysconfig\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661122    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysconfig\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661125    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-host\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661152    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661161    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-host\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661159    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/66b62ff9-9859-419a-9aa0-365b4f25229b-konnectivity-ca\") pod \"konnectivity-agent-kd766\" (UID: \"66b62ff9-9859-419a-9aa0-365b4f25229b\") " pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.661369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcnng\" (UniqueName: \"kubernetes.io/projected/e5de8057-31c1-410d-9f8e-f5479e1c644a-kube-api-access-fcnng\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661201    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661226    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-socket-dir-parent\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-hostroot\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-etc-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661308    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-node-log\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661343    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-log-socket\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661354    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-etc-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661373    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-node-log\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661385    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-log-socket\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661360    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65032d2e-e6b0-4469-9921-3273ef4e5f75-serviceca\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661409    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-kubernetes\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-sys\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661450    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/486b3034-61ca-49ad-9061-bf7c9369447c-dbus\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661482    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-var-lib-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661490    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-kubernetes\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.662298 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661498    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661499    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-sys\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661545    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-var-lib-openvswitch\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661562    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/486b3034-61ca-49ad-9061-bf7c9369447c-dbus\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661664    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-cni-bin\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661632    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-conf-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-lib-modules\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pcs\" (UniqueName: \"kubernetes.io/projected/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-kube-api-access-86pcs\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661838    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gvp2\" (UniqueName: \"kubernetes.io/projected/524b3e5f-fe27-42de-b73e-a0eea84945bf-kube-api-access-8gvp2\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661880    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/12276eee-8052-47e3-83b1-f1ba413727e2-multus-daemon-config\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.661938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-tuned\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662000    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-etc-selinux\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662030    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67cp4\" (UniqueName: \"kubernetes.io/projected/7ee5adca-42ed-4f10-b33c-c05a981388f9-kube-api-access-67cp4\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662035    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-lib-modules\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663053 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662079    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12276eee-8052-47e3-83b1-f1ba413727e2-cni-binary-copy\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662104    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-etc-selinux\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysctl-d\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662172    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-sysctl-d\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662272    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662341    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-systemd\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-cni-bin\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662422    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-run-systemd\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662470    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-k8s-cni-cncf-io\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662501    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-env-overrides\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662542    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-cni-bin\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662547    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d2ss\" (UniqueName: \"kubernetes.io/projected/65032d2e-e6b0-4469-9921-3273ef4e5f75-kube-api-access-4d2ss\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662598    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-system-cni-dir\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662623    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-cnibin\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-os-release\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-kubelet-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-socket-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.663801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-registration-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662782    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-multus-certs\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.662808    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovnkube-config\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663214    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-env-overrides\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663284    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-kubelet-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663387    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-socket-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663444    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01e967d7-7ae9-4bd1-a51c-74608439db60-registration-dir\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-systemd\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663843    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-var-lib-kubelet\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663892    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/486b3034-61ca-49ad-9061-bf7c9369447c-kubelet-config\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663922    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovnkube-config\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.663961    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-run-netns\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664011    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-cni-netd\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664020    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-run-netns\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664029    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-tmp\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5de8057-31c1-410d-9f8e-f5479e1c644a-iptables-alerter-script\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664030    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/486b3034-61ca-49ad-9061-bf7c9369447c-kubelet-config\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:32.664457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664069    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-system-cni-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664070    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/524b3e5f-fe27-42de-b73e-a0eea84945bf-host-cni-netd\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664107    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-var-lib-kubelet\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664144    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-cni-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664244    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/524b3e5f-fe27-42de-b73e-a0eea84945bf-ovn-node-metrics-cert\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664297    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-systemd\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664310    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-etc-tuned\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664324    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/66b62ff9-9859-419a-9aa0-365b4f25229b-agent-certs\") pod \"konnectivity-agent-kd766\" (UID: \"66b62ff9-9859-419a-9aa0-365b4f25229b\") " pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.664922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.664499    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5de8057-31c1-410d-9f8e-f5479e1c644a-iptables-alerter-script\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.671521 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.671499    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:54:32.671641 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.671526    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:54:32.671641 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.671539    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:32.671641 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.671598    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:33.171580131 +0000 UTC m=+2.018608102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:32.673532 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.673492    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkth8\" (UniqueName: \"kubernetes.io/projected/e1fd811b-ecd4-4720-b927-556e91ace214-kube-api-access-xkth8\") pod \"node-resolver-b8bd9\" (UID: \"e1fd811b-ecd4-4720-b927-556e91ace214\") " pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.674140 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.674114    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gvp2\" (UniqueName: \"kubernetes.io/projected/524b3e5f-fe27-42de-b73e-a0eea84945bf-kube-api-access-8gvp2\") pod \"ovnkube-node-n5zzg\" (UID: \"524b3e5f-fe27-42de-b73e-a0eea84945bf\") " pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.674227 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.674170    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcnng\" (UniqueName: \"kubernetes.io/projected/e5de8057-31c1-410d-9f8e-f5479e1c644a-kube-api-access-fcnng\") pod \"iptables-alerter-prtsv\" (UID: \"e5de8057-31c1-410d-9f8e-f5479e1c644a\") " pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.674590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.674571    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpk7q\" (UniqueName: \"kubernetes.io/projected/01e967d7-7ae9-4bd1-a51c-74608439db60-kube-api-access-wpk7q\") pod \"aws-ebs-csi-driver-node-84nnt\" (UID: \"01e967d7-7ae9-4bd1-a51c-74608439db60\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.674674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.674623    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ll9d\" (UniqueName: \"kubernetes.io/projected/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-kube-api-access-8ll9d\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:32.675412 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.675393    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86pcs\" (UniqueName: \"kubernetes.io/projected/eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6-kube-api-access-86pcs\") pod \"tuned-gmw5z\" (UID: \"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6\") " pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.681783 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.681759    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:54:32.765156 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-system-cni-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765161    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-cni-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765186    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65032d2e-e6b0-4469-9921-3273ef4e5f75-host\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-kubelet\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765266    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-cni-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765284    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-os-release\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765270    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-system-cni-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765324    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-netns\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765349    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-os-release\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765361    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-kubelet\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765364    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-netns\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765369    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-cni-multus\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765396    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65032d2e-e6b0-4469-9921-3273ef4e5f75-host\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765412    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-cni-multus\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765424    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn4jb\" (UniqueName: \"kubernetes.io/projected/12276eee-8052-47e3-83b1-f1ba413727e2-kube-api-access-xn4jb\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765462    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-cnibin\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-etc-kubernetes\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765548    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765576    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-socket-dir-parent\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765600    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-hostroot\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765627    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65032d2e-e6b0-4469-9921-3273ef4e5f75-serviceca\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765627    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-cnibin\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-cni-bin\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765678    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-etc-kubernetes\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.765804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765709    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765709    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-conf-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765749    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765756    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-var-lib-cni-bin\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/12276eee-8052-47e3-83b1-f1ba413727e2-multus-daemon-config\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765772    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-conf-dir\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765773    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-multus-socket-dir-parent\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765794    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-hostroot\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765825    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.765788    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67cp4\" (UniqueName: \"kubernetes.io/projected/7ee5adca-42ed-4f10-b33c-c05a981388f9-kube-api-access-67cp4\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766068    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12276eee-8052-47e3-83b1-f1ba413727e2-cni-binary-copy\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766123    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-k8s-cni-cncf-io\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766150    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d2ss\" (UniqueName: \"kubernetes.io/projected/65032d2e-e6b0-4469-9921-3273ef4e5f75-kube-api-access-4d2ss\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-system-cni-dir\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766199    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-cnibin\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-os-release\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766250    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-multus-certs\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.766699 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766267    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65032d2e-e6b0-4469-9921-3273ef4e5f75-serviceca\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766274    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766248    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7ee5adca-42ed-4f10-b33c-c05a981388f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766329    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-k8s-cni-cncf-io\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766330    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-system-cni-dir\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766688    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/12276eee-8052-47e3-83b1-f1ba413727e2-host-run-multus-certs\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766759    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-os-release\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766911    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/12276eee-8052-47e3-83b1-f1ba413727e2-multus-daemon-config\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.766940    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ee5adca-42ed-4f10-b33c-c05a981388f9-cnibin\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.767497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.767255    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12276eee-8052-47e3-83b1-f1ba413727e2-cni-binary-copy\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.786014 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.785986    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d2ss\" (UniqueName: \"kubernetes.io/projected/65032d2e-e6b0-4469-9921-3273ef4e5f75-kube-api-access-4d2ss\") pod \"node-ca-kkdj4\" (UID: \"65032d2e-e6b0-4469-9921-3273ef4e5f75\") " pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.791284 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.791258    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67cp4\" (UniqueName: \"kubernetes.io/projected/7ee5adca-42ed-4f10-b33c-c05a981388f9-kube-api-access-67cp4\") pod \"multus-additional-cni-plugins-xt7ss\" (UID: \"7ee5adca-42ed-4f10-b33c-c05a981388f9\") " pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.798976 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.798929    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn4jb\" (UniqueName: \"kubernetes.io/projected/12276eee-8052-47e3-83b1-f1ba413727e2-kube-api-access-xn4jb\") pod \"multus-psd4s\" (UID: \"12276eee-8052-47e3-83b1-f1ba413727e2\") " pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.870886 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.870802    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b8bd9"
Jul 08 06:54:32.876236 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.876209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:32.883616 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.883582    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1ff6d8fbf84e1b6a71573c64b922f8.slice/crio-f1ebd42a3b6369c2ce8e6051988b83a8ffecb14d8735d6321e4709ab33cd749c WatchSource:0}: Error finding container f1ebd42a3b6369c2ce8e6051988b83a8ffecb14d8735d6321e4709ab33cd749c: Status 404 returned error can't find the container with id f1ebd42a3b6369c2ce8e6051988b83a8ffecb14d8735d6321e4709ab33cd749c
Jul 08 06:54:32.884164 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.884147    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fd811b_ecd4_4720_b927_556e91ace214.slice/crio-07e84085db4e3418cdcaf0ed4c3a439f0030abf5d6c30533cd10ebbb8b3d284a WatchSource:0}: Error finding container 07e84085db4e3418cdcaf0ed4c3a439f0030abf5d6c30533cd10ebbb8b3d284a: Status 404 returned error can't find the container with id 07e84085db4e3418cdcaf0ed4c3a439f0030abf5d6c30533cd10ebbb8b3d284a
Jul 08 06:54:32.885188 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.885169    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ad56ce288e22a2bd7487c8f13b9062.slice/crio-a0d7d98d36fca9a1f97c5613211c8a10ade433b4a5e56ab376e025c75dbac9d3 WatchSource:0}: Error finding container a0d7d98d36fca9a1f97c5613211c8a10ade433b4a5e56ab376e025c75dbac9d3: Status 404 returned error can't find the container with id a0d7d98d36fca9a1f97c5613211c8a10ade433b4a5e56ab376e025c75dbac9d3
Jul 08 06:54:32.885929 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.885914    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b62ff9_9859_419a_9aa0_365b4f25229b.slice/crio-c904633386db0752e755b20fc259159c99448470434220f3ba1543feabc32d1f WatchSource:0}: Error finding container c904633386db0752e755b20fc259159c99448470434220f3ba1543feabc32d1f: Status 404 returned error can't find the container with id c904633386db0752e755b20fc259159c99448470434220f3ba1543feabc32d1f
Jul 08 06:54:32.889941 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.889847    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gmw5z"
Jul 08 06:54:32.889941 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.889916    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 06:54:32.898687 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.898668    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4a9eaa_47a0_4abf_aac5_abe6781cc9f6.slice/crio-76cb35dc55834ed96d6b7ca88839f431ed90edde69769dad3f5c1dddaf13e171 WatchSource:0}: Error finding container 76cb35dc55834ed96d6b7ca88839f431ed90edde69769dad3f5c1dddaf13e171: Status 404 returned error can't find the container with id 76cb35dc55834ed96d6b7ca88839f431ed90edde69769dad3f5c1dddaf13e171
Jul 08 06:54:32.904443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.904424    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-prtsv"
Jul 08 06:54:32.910008 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.909932    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:54:32.911852 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.911735    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5de8057_31c1_410d_9f8e_f5479e1c644a.slice/crio-ead276d4af509374e6794f0b4c0a2d744ed168c63e7dc522b0f68d348a0467dc WatchSource:0}: Error finding container ead276d4af509374e6794f0b4c0a2d744ed168c63e7dc522b0f68d348a0467dc: Status 404 returned error can't find the container with id ead276d4af509374e6794f0b4c0a2d744ed168c63e7dc522b0f68d348a0467dc
Jul 08 06:54:32.919623 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.919604    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524b3e5f_fe27_42de_b73e_a0eea84945bf.slice/crio-9bcb6af61f9d86eda564b8434a9cd8c41482cc3bedc070720cb90aac657e899d WatchSource:0}: Error finding container 9bcb6af61f9d86eda564b8434a9cd8c41482cc3bedc070720cb90aac657e899d: Status 404 returned error can't find the container with id 9bcb6af61f9d86eda564b8434a9cd8c41482cc3bedc070720cb90aac657e899d
Jul 08 06:54:32.932655 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.932638    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt"
Jul 08 06:54:32.939822 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.939799    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e967d7_7ae9_4bd1_a51c_74608439db60.slice/crio-86632859e89c97d69ca26f5d0ea3fee2e8969623296e6bade9188399153ec689 WatchSource:0}: Error finding container 86632859e89c97d69ca26f5d0ea3fee2e8969623296e6bade9188399153ec689: Status 404 returned error can't find the container with id 86632859e89c97d69ca26f5d0ea3fee2e8969623296e6bade9188399153ec689
Jul 08 06:54:32.945877 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.945861    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kkdj4"
Jul 08 06:54:32.952555 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.952530    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65032d2e_e6b0_4469_9921_3273ef4e5f75.slice/crio-4aa0b0bc9ee3e0293cabd75b0f0188fbc77fc8a6954522b466550b1f32833bc5 WatchSource:0}: Error finding container 4aa0b0bc9ee3e0293cabd75b0f0188fbc77fc8a6954522b466550b1f32833bc5: Status 404 returned error can't find the container with id 4aa0b0bc9ee3e0293cabd75b0f0188fbc77fc8a6954522b466550b1f32833bc5
Jul 08 06:54:32.975893 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.975873    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xt7ss"
Jul 08 06:54:32.982357 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.982337    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee5adca_42ed_4f10_b33c_c05a981388f9.slice/crio-bf174433c08cbfc4c019f9fdcec5b7cbccc2aa37d88881abc6a5717698762811 WatchSource:0}: Error finding container bf174433c08cbfc4c019f9fdcec5b7cbccc2aa37d88881abc6a5717698762811: Status 404 returned error can't find the container with id bf174433c08cbfc4c019f9fdcec5b7cbccc2aa37d88881abc6a5717698762811
Jul 08 06:54:32.988316 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:32.988301    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-psd4s"
Jul 08 06:54:32.994871 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:54:32.994850    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12276eee_8052_47e3_83b1_f1ba413727e2.slice/crio-7ea3ec8e65668bd6c8a149e5691292cc18774c98a5437baeda962c4b3a844059 WatchSource:0}: Error finding container 7ea3ec8e65668bd6c8a149e5691292cc18774c98a5437baeda962c4b3a844059: Status 404 returned error can't find the container with id 7ea3ec8e65668bd6c8a149e5691292cc18774c98a5437baeda962c4b3a844059
Jul 08 06:54:32.996399 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.996381    2571 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-psd4s_openshift-multus(12276eee-8052-47e3-83b1-f1ba413727e2): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 08 06:54:32.996450 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:32.996418    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-psd4s" podUID="12276eee-8052-47e3-83b1-f1ba413727e2"
Jul 08 06:54:33.168774 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:33.168695    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:33.168774 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:33.168728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:33.168933 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:33.168836    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:33.168933 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:33.168843    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:33.168933 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:33.168887    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:54:34.168873875 +0000 UTC m=+3.015901822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:33.168933 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:33.168900    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:34.168894645 +0000 UTC m=+3.015922595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:34.517184 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.517155    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:34.517347 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:34.517296    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:54:34.517347 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:34.517317    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:54:34.517347 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:34.517327    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:34.517466 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:34.517375    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:35.517362986 +0000 UTC m=+3.116652670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:34.861523 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.861430    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 06:49:33 +0000 UTC" deadline="2027-12-07 13:23:50.933102905 +0000 UTC"
Jul 08 06:54:34.861523 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.861471    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12414h29m16.07163735s"
Jul 08 06:54:34.915085 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.915060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:34.915247 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:34.915178    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:34.923863 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.923810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kkdj4" event={"ID":"65032d2e-e6b0-4469-9921-3273ef4e5f75","Type":"ContainerStarted","Data":"4aa0b0bc9ee3e0293cabd75b0f0188fbc77fc8a6954522b466550b1f32833bc5"}
Jul 08 06:54:34.938008 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.937978    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kd766" event={"ID":"66b62ff9-9859-419a-9aa0-365b4f25229b","Type":"ContainerStarted","Data":"c904633386db0752e755b20fc259159c99448470434220f3ba1543feabc32d1f"}
Jul 08 06:54:34.945800 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.945772    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b8bd9" event={"ID":"e1fd811b-ecd4-4720-b927-556e91ace214","Type":"ContainerStarted","Data":"07e84085db4e3418cdcaf0ed4c3a439f0030abf5d6c30533cd10ebbb8b3d284a"}
Jul 08 06:54:34.953667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.953606    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal" event={"ID":"3e1ff6d8fbf84e1b6a71573c64b922f8","Type":"ContainerStarted","Data":"f1ebd42a3b6369c2ce8e6051988b83a8ffecb14d8735d6321e4709ab33cd749c"}
Jul 08 06:54:34.966789 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.966606    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-psd4s" event={"ID":"12276eee-8052-47e3-83b1-f1ba413727e2","Type":"ContainerStarted","Data":"7ea3ec8e65668bd6c8a149e5691292cc18774c98a5437baeda962c4b3a844059"}
Jul 08 06:54:34.968672 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:34.968544    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:55ec7147f859301bbb8288966c8a21c5db729200f3c3d91db671978baef79517\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-psd4s" podUID="12276eee-8052-47e3-83b1-f1ba413727e2"
Jul 08 06:54:34.970086 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.970026    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerStarted","Data":"bf174433c08cbfc4c019f9fdcec5b7cbccc2aa37d88881abc6a5717698762811"}
Jul 08 06:54:34.977734 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.977652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt" event={"ID":"01e967d7-7ae9-4bd1-a51c-74608439db60","Type":"ContainerStarted","Data":"86632859e89c97d69ca26f5d0ea3fee2e8969623296e6bade9188399153ec689"}
Jul 08 06:54:34.985780 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.985629    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"9bcb6af61f9d86eda564b8434a9cd8c41482cc3bedc070720cb90aac657e899d"}
Jul 08 06:54:34.992385 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:34.992243    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-prtsv" event={"ID":"e5de8057-31c1-410d-9f8e-f5479e1c644a","Type":"ContainerStarted","Data":"ead276d4af509374e6794f0b4c0a2d744ed168c63e7dc522b0f68d348a0467dc"}
Jul 08 06:54:35.004479 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.004423    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gmw5z" event={"ID":"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6","Type":"ContainerStarted","Data":"76cb35dc55834ed96d6b7ca88839f431ed90edde69769dad3f5c1dddaf13e171"}
Jul 08 06:54:35.018383 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.018330    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal" event={"ID":"c8ad56ce288e22a2bd7487c8f13b9062","Type":"ContainerStarted","Data":"a0d7d98d36fca9a1f97c5613211c8a10ade433b4a5e56ab376e025c75dbac9d3"}
Jul 08 06:54:35.425192 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.425157    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:35.425381 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.425206    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:35.425480 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.425383    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:35.425480 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.425445    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:54:37.425426806 +0000 UTC m=+5.024716502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:35.425906 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.425885    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:35.426001 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.425943    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:37.42592717 +0000 UTC m=+5.025216867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:35.525773 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.525739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:35.525956 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.525927    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:54:35.525956 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.525946    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:54:35.525956 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.525956    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:35.526098 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.526025    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:37.526005975 +0000 UTC m=+5.125295671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:35.862558 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.862425    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 06:49:33 +0000 UTC" deadline="2028-03-30 15:52:08.937319744 +0000 UTC"
Jul 08 06:54:35.862558 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.862486    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15152h57m33.074836593s"
Jul 08 06:54:35.908497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.908066    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:35.908497 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.908205    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:35.908742 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:35.908581    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:35.908742 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:35.908673    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:36.167282 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:36.167249    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 06:54:36.916894 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:36.916352    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:36.916894 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:36.916463    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:37.441953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:37.441918    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:37.442151 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:37.441964    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:37.442223 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.442163    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:37.442275 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.442226    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:54:41.442207907 +0000 UTC m=+9.041497606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:37.442748 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.442728    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:37.442863 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.442782    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:41.442767118 +0000 UTC m=+9.042056800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:37.543052 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:37.543016    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:37.543223 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.543162    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:54:37.543223 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.543185    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:54:37.543223 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.543197    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:37.543373 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.543253    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:41.543234868 +0000 UTC m=+9.142524560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:37.907966 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:37.907886    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:37.908116 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.908024    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:37.908436 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:37.908419    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:37.908528 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:37.908510    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:38.911106 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:38.911081    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:38.911549 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:38.911196    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:39.908150 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:39.908118    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:39.908334 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:39.908169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:39.908334 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:39.908285    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:39.908439 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:39.908416    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:40.908361 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:40.908320    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:40.908840 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:40.908446    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:41.475237 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:41.475083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:41.475237 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:41.475136    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:41.475454 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.475258    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:41.475454 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.475261    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:41.475454 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.475327    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:54:49.475311431 +0000 UTC m=+17.074601120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:41.475454 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.475342    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:49.475335771 +0000 UTC m=+17.074625451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:41.576309 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:41.576271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:41.576487 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.576426    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:54:41.576487 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.576446    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:54:41.576487 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.576457    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:41.576614 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.576509    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:54:49.576496574 +0000 UTC m=+17.175786255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:41.908533 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:41.908438    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:41.908950 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.908605    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:41.908950 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:41.908452    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:41.908950 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:41.908860    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:42.908151 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:42.908113    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:42.908321 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:42.908228    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:43.907735 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:43.907701    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:43.908201 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:43.907754    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:43.908201 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:43.907817    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:43.908201 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:43.907912    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:44.911031 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:44.910959    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:44.911429 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:44.911053    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:45.908252 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:45.908213    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:45.908252 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:45.908236    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:45.908501 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:45.908366    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:45.908754 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:45.908720    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:46.910168 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:46.910136    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:46.910574 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:46.910242    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:47.908345 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:47.908306    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:47.908542 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:47.908444    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:47.908542 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:47.908509    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:47.908654 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:47.908635    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:48.909828 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:48.909800    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:48.910170 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:48.909910    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:49.532791 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:49.532758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:49.532931 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:49.532808    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:49.532931 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.532924    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:49.533050 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.532984    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:55:05.532964277 +0000 UTC m=+33.132253972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:54:49.533373 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.533352    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:49.533470 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.533408    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:05.533393277 +0000 UTC m=+33.132682958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:54:49.633517 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:49.633496    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:49.633738 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.633720    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:54:49.633808 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.633746    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:54:49.633808 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.633758    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:49.633913 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.633810    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:05.633794174 +0000 UTC m=+33.233083860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:54:49.908177 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:49.907996    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:49.908296 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:49.908052    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:49.908343 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.908286    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:49.908376 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:49.908345    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:50.055950 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.055920    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ee5adca-42ed-4f10-b33c-c05a981388f9" containerID="c18f8ae14e0a985dfee6edc92e27b636d7d9c1a1d12d08c226f0b89e218b86be" exitCode=0
Jul 08 06:54:50.056843 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.055965    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerDied","Data":"c18f8ae14e0a985dfee6edc92e27b636d7d9c1a1d12d08c226f0b89e218b86be"}
Jul 08 06:54:50.058193 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.058156    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt" event={"ID":"01e967d7-7ae9-4bd1-a51c-74608439db60","Type":"ContainerStarted","Data":"7f70d378897bfa30b490235ca9bb53d37e1a2a8afcc99c476d1ee332e6ca1dd4"}
Jul 08 06:54:50.061621 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.061600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"07478a53a3c2801185b2536236a1cb8c7dc9785eab7e291bbd96e76efa8b2720"}
Jul 08 06:54:50.061765 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.061748    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"5ca4f288051d087fbf4bc85cb494d2e5623ac110605a8a57a6fafdcf2c36d152"}
Jul 08 06:54:50.061841 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.061808    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"86306c4988aaee798af58615ad022a9fd18374e301a774f9b5231debf76ee7f3"}
Jul 08 06:54:50.061895 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.061859    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"9429aee9e42c4a171529ce7dadc4a706aa31f26c091dbb1b83cbdbd739a4ed79"}
Jul 08 06:54:50.061952 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.061912    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"d91cb77f6d81306d0f4a2bd3ff7081e57f39626b6e43931416a9b000bf9ac404"}
Jul 08 06:54:50.062012 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.061961    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"ef50dbd58b91c3ea333c266e7d73e2d46464a328fa70ffadcf0aa1487d8ba42d"}
Jul 08 06:54:50.063245 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.063198    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gmw5z" event={"ID":"eb4a9eaa-47a0-4abf-aac5-abe6781cc9f6","Type":"ContainerStarted","Data":"b665984dd449c431d1b58fd61bdd505e56682626fa26637e22c75f627cd726af"}
Jul 08 06:54:50.064845 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.064821    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal" event={"ID":"c8ad56ce288e22a2bd7487c8f13b9062","Type":"ContainerStarted","Data":"418f45fb6928e582048be8caebb5adcdae80e8d36097656523457ccaa0eef926"}
Jul 08 06:54:50.067110 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.066802    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kkdj4" event={"ID":"65032d2e-e6b0-4469-9921-3273ef4e5f75","Type":"ContainerStarted","Data":"4b5054c5ab291431f9881509979fd267c5e4024373ca6c828116f08839bf7e98"}
Jul 08 06:54:50.068393 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.068372    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kd766" event={"ID":"66b62ff9-9859-419a-9aa0-365b4f25229b","Type":"ContainerStarted","Data":"395de4081b595b20a772a83d94253ebad63c5bcb7be040668ddbb1cb5d061b26"}
Jul 08 06:54:50.070227 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.070203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b8bd9" event={"ID":"e1fd811b-ecd4-4720-b927-556e91ace214","Type":"ContainerStarted","Data":"cfd28772e8a33e53dea9585a0dbab07eba3aa8cd2ce66ec5dfae41c7f41616fc"}
Jul 08 06:54:50.072002 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.071976    2571 generic.go:356] "Generic (PLEG): container finished" podID="3e1ff6d8fbf84e1b6a71573c64b922f8" containerID="3cc47c52e2164aba6ad5084acd4059e35a1e983775745d6caf8a93e344e41977" exitCode=0
Jul 08 06:54:50.072117 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.072017    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal" event={"ID":"3e1ff6d8fbf84e1b6a71573c64b922f8","Type":"ContainerDied","Data":"3cc47c52e2164aba6ad5084acd4059e35a1e983775745d6caf8a93e344e41977"}
Jul 08 06:54:50.096745 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.096501    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-54.ec2.internal" podStartSLOduration=17.096487554 podStartE2EDuration="17.096487554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:50.095905401 +0000 UTC m=+17.695195107" watchObservedRunningTime="2026-07-08 06:54:50.096487554 +0000 UTC m=+17.695777256"
Jul 08 06:54:50.132372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.132319    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gmw5z" podStartSLOduration=2.050053819 podStartE2EDuration="17.132299117s" totalImagesPullingTime="15.082245298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:50.115922343 +0000 UTC m=+17.715212046" watchObservedRunningTime="2026-07-08 06:54:50.132299117 +0000 UTC m=+17.731588823"
Jul 08 06:54:50.133372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.133325    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kkdj4" podStartSLOduration=2.105038155 podStartE2EDuration="17.133316446s" totalImagesPullingTime="15.028278291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:50.132977972 +0000 UTC m=+17.732267676" watchObservedRunningTime="2026-07-08 06:54:50.133316446 +0000 UTC m=+17.732606150"
Jul 08 06:54:50.188965 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.188286    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b8bd9" podStartSLOduration=3.057256066 podStartE2EDuration="18.188271789s" totalImagesPullingTime="15.131015723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:50.188101645 +0000 UTC m=+17.787391350" watchObservedRunningTime="2026-07-08 06:54:50.188271789 +0000 UTC m=+17.787561492"
Jul 08 06:54:50.209944 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.209835    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kd766" podStartSLOduration=2.117914074 podStartE2EDuration="17.209816083s" totalImagesPullingTime="15.091902009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:50.20919747 +0000 UTC m=+17.808487173" watchObservedRunningTime="2026-07-08 06:54:50.209816083 +0000 UTC m=+17.809105787"
Jul 08 06:54:50.880997 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.880972    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 06:54:50.912115 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:50.912091    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:50.912243 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:50.912198    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:51.076922 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.076873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt" event={"ID":"01e967d7-7ae9-4bd1-a51c-74608439db60","Type":"ContainerStarted","Data":"392409b10699d9e34783146677830a3562b7590c3aa9bece561ae61e16801cfd"}
Jul 08 06:54:51.078853 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.078822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-prtsv" event={"ID":"e5de8057-31c1-410d-9f8e-f5479e1c644a","Type":"ContainerStarted","Data":"2cfdbc5067510346eccf73383aa0ff17b98c802be2228f28762aad13b5a97660"}
Jul 08 06:54:51.080933 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.080905    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal" event={"ID":"3e1ff6d8fbf84e1b6a71573c64b922f8","Type":"ContainerStarted","Data":"2b05edbc3266a70bff7d04ae07f0c1b83bca73f16463fc417e9acc9d901d2e88"}
Jul 08 06:54:51.121821 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.121766    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-54.ec2.internal" podStartSLOduration=18.121747327 podStartE2EDuration="18.121747327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:51.121562579 +0000 UTC m=+18.720852283" watchObservedRunningTime="2026-07-08 06:54:51.121747327 +0000 UTC m=+18.721037033"
Jul 08 06:54:51.122503 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.122438    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-prtsv" podStartSLOduration=3.053195149 podStartE2EDuration="18.122429454s" totalImagesPullingTime="15.069234305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:51.098597952 +0000 UTC m=+18.697887656" watchObservedRunningTime="2026-07-08 06:54:51.122429454 +0000 UTC m=+18.721719158"
Jul 08 06:54:51.871729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.871064    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T06:54:50.880993684Z","UUID":"79bd2965-805f-4a3f-a821-8e2906a29638","Handler":null,"Name":"","Endpoint":""}
Jul 08 06:54:51.873071 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.872592    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 06:54:51.873071 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.872633    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 06:54:51.907905 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.907869    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:51.908032 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:51.907886    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:51.908081 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:51.908030    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:51.908081 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:51.908055    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:52.086110 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:52.086083    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt" event={"ID":"01e967d7-7ae9-4bd1-a51c-74608439db60","Type":"ContainerStarted","Data":"7436bf6ef4fa819212b5b10cbcd17096e47b598a2efdaf2363aa6f445707d3fb"}
Jul 08 06:54:52.612852 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:52.612819    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:52.613524 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:52.613508    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:52.630753 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:52.630676    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84nnt" podStartSLOduration=2.738512223 podStartE2EDuration="19.630657449s" totalImagesPullingTime="16.892145226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:54:52.109962264 +0000 UTC m=+19.709251968" watchObservedRunningTime="2026-07-08 06:54:52.630657449 +0000 UTC m=+20.229947154"
Jul 08 06:54:52.916209 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:52.916134    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:52.916358 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:52.916250    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:53.093098 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:53.093058    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"2afb3971fb7fdb0a663cf0aa263570237de6040310823f1b3b84f3f80398284c"}
Jul 08 06:54:53.093518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:53.093424    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:53.093908 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:53.093874    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kd766"
Jul 08 06:54:53.907881 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:53.907847    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:53.908066 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:53.907977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:53.908127 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:53.908097    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:53.908127 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:53.907973    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:54.908302 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:54.908268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:54.908873 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:54.908399    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:55.907620 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:55.907587    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:55.907898 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:55.907587    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:55.907898 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:55.907725    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:55.907898 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:55.907874    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:56.907898 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:56.907863    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:56.908361 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:56.907997    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:57.907478 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:57.907442    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:57.907662 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:57.907574    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:57.907662 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:57.907627    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:57.907782 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:57.907763    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:54:58.908218 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:58.908180    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:54:58.908627 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:58.908321    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:54:59.907497 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:59.907389    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:54:59.907605 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:54:59.907450    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:54:59.907668 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:59.907565    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:54:59.907761 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:54:59.907736    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:55:00.113334 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.113291    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" event={"ID":"524b3e5f-fe27-42de-b73e-a0eea84945bf","Type":"ContainerStarted","Data":"73b4ab9beff0b0778ec7369f36a703453b98de90c7225e1ffadf35c4652ac29e"}
Jul 08 06:55:00.114137 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.113770    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:55:00.114137 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.113845    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:55:00.114137 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.113894    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:55:00.115575 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.115543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-psd4s" event={"ID":"12276eee-8052-47e3-83b1-f1ba413727e2","Type":"ContainerStarted","Data":"7524e2e2a28d07945d76272a6d45432995281081f5ce96cdb8d0dbba4b799e4e"}
Jul 08 06:55:00.118081 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.118046    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerStarted","Data":"211c42ed5c78e1ce215eb5810fb5975277619fbc12442bf5995fb5083c1cf3d1"}
Jul 08 06:55:00.131110 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.131087    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:55:00.133926 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.133907    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:55:00.147209 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.147144    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg" podStartSLOduration=11.810735236 podStartE2EDuration="27.147131502s" totalImagesPullingTime="15.336396266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:00.146750758 +0000 UTC m=+27.746040461" watchObservedRunningTime="2026-07-08 06:55:00.147131502 +0000 UTC m=+27.746421204"
Jul 08 06:55:00.205286 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.205238    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-psd4s" podStartSLOduration=1.4995003900000001 podStartE2EDuration="27.205222459s" totalImagesPullingTime="25.705722069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-08 06:55:00.171863819 +0000 UTC m=+27.771153523" watchObservedRunningTime="2026-07-08 06:55:00.205222459 +0000 UTC m=+27.804512162"
Jul 08 06:55:00.907801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:00.907777    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:00.907917 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:00.907879    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:55:01.121779 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:01.121744    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ee5adca-42ed-4f10-b33c-c05a981388f9" containerID="211c42ed5c78e1ce215eb5810fb5975277619fbc12442bf5995fb5083c1cf3d1" exitCode=0
Jul 08 06:55:01.122336 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:01.121831    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerDied","Data":"211c42ed5c78e1ce215eb5810fb5975277619fbc12442bf5995fb5083c1cf3d1"}
Jul 08 06:55:01.908799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:01.908542    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:01.909105 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:01.908910    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:55:01.909105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:01.908543    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:55:01.909105 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:01.909026    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:55:02.134891 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:02.134861    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6qqpr"]
Jul 08 06:55:02.135322 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:02.134971    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:02.135322 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:02.135071    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:55:02.139013 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:02.138982    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vtskv"]
Jul 08 06:55:02.139143 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:02.139097    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:55:02.139228 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:02.139211    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:55:02.141403 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:02.140360    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lb9c2"]
Jul 08 06:55:02.141403 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:02.140437    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:02.141403 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:02.140513    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:55:03.128325 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:03.128287    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ee5adca-42ed-4f10-b33c-c05a981388f9" containerID="b9e547fa32f8490bda7a944cb2b0ecef087a8db7629aeb0c7ad93bfe83a70141" exitCode=0
Jul 08 06:55:03.128511 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:03.128332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerDied","Data":"b9e547fa32f8490bda7a944cb2b0ecef087a8db7629aeb0c7ad93bfe83a70141"}
Jul 08 06:55:03.907961 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:03.907925    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:03.908332 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:03.907924    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:55:03.908332 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:03.908040    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qqpr" podUID="486b3034-61ca-49ad-9061-bf7c9369447c"
Jul 08 06:55:03.908332 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:03.908138    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vtskv" podUID="2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624"
Jul 08 06:55:03.908332 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:03.907924    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:03.908332 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:03.908201    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lb9c2" podUID="95e16ab3-b354-4c20-a5fe-d8039ed13111"
Jul 08 06:55:04.132526 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:04.132491    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ee5adca-42ed-4f10-b33c-c05a981388f9" containerID="ef20a1db4d310e90f1179331cfa7a28d8ef9a5a45449f364fa96bf0c2ab9917c" exitCode=0
Jul 08 06:55:04.132718 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:04.132535    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerDied","Data":"ef20a1db4d310e90f1179331cfa7a28d8ef9a5a45449f364fa96bf0c2ab9917c"}
Jul 08 06:55:05.549512 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.549309    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:55:05.549979 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.549527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:05.549979 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.549472    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:05.549979 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.549642    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:37.549623671 +0000 UTC m=+65.148913372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 06:55:05.549979 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.549645    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:05.549979 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.549701    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret podName:486b3034-61ca-49ad-9061-bf7c9369447c nodeName:}" failed. No retries permitted until 2026-07-08 06:55:37.54967108 +0000 UTC m=+65.148960762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret") pod "global-pull-secret-syncer-6qqpr" (UID: "486b3034-61ca-49ad-9061-bf7c9369447c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 06:55:05.650704 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.650603    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:05.650874 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.650756    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 06:55:05.650874 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.650774    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 06:55:05.650874 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.650787    2571 projected.go:196] Error preparing data for projected volume kube-api-access-ncgvc for pod openshift-network-diagnostics/network-check-target-lb9c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:05.650874 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.650845    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc podName:95e16ab3-b354-4c20-a5fe-d8039ed13111 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:37.650826845 +0000 UTC m=+65.250116537 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ncgvc" (UniqueName: "kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc") pod "network-check-target-lb9c2" (UID: "95e16ab3-b354-4c20-a5fe-d8039ed13111") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 06:55:05.651363 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.651342    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-54.ec2.internal" event="NodeReady"
Jul 08 06:55:05.651510 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.651489    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 06:55:05.695039 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.695008    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bd9657ff9-lv66b"]
Jul 08 06:55:05.699018 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.698990    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.702385 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.702364    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 06:55:05.702498 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.702368    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 06:55:05.703124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.702968    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 06:55:05.703124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.703054    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q8mxm\""
Jul 08 06:55:05.708806 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.708787    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 06:55:05.714031 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.714010    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bd9657ff9-lv66b"]
Jul 08 06:55:05.714592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.714571    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g468f"]
Jul 08 06:55:05.718461 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.718445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.721840 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.721819    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 06:55:05.722096 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.722077    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 06:55:05.722402 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.722375    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6zr5\""
Jul 08 06:55:05.723809 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.723791    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fm69n"]
Jul 08 06:55:05.727587 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.727569    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.730783 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.730615    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 06:55:05.730783 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.730644    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 06:55:05.731043 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.731011    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fs5kh\""
Jul 08 06:55:05.731119 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.731016    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 06:55:05.732719 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.731798    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g468f"]
Jul 08 06:55:05.737374 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.737322    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fm69n"]
Jul 08 06:55:05.851960 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.851923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-trusted-ca\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.851964    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd8rk\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-kube-api-access-pd8rk\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.851993    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852018    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-certificates\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-bound-sa-token\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852081    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.852132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852115    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvxc\" (UniqueName: \"kubernetes.io/projected/e96bc4c3-a102-47da-8a20-fa8c17a05c83-kube-api-access-srvxc\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.852372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852151    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f46102e8-813f-45bd-bcad-cac9e30b7d70-cert\") pod \"ingress-canary-fm69n\" (UID: \"f46102e8-813f-45bd-bcad-cac9e30b7d70\") " pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.852372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852225    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-image-registry-private-configuration\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-installation-pull-secrets\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e96bc4c3-a102-47da-8a20-fa8c17a05c83-config-volume\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.852372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852334    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96bc4c3-a102-47da-8a20-fa8c17a05c83-tmp-dir\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.852372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4077e488-ca01-4396-9f45-cebd5fbdcef1-ca-trust-extracted\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.852602 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.852429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dqm6\" (UniqueName: \"kubernetes.io/projected/f46102e8-813f-45bd-bcad-cac9e30b7d70-kube-api-access-5dqm6\") pod \"ingress-canary-fm69n\" (UID: \"f46102e8-813f-45bd-bcad-cac9e30b7d70\") " pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.894549 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.894508    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kd766_66b62ff9-9859-419a-9aa0-365b4f25229b/konnectivity-agent/0.log" limit=1024
Jul 08 06:55:05.907513 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.907447    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:05.907620 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.907558    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:55:05.907763 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.907746    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:05.910996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.910939    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 06:55:05.911122 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.911068    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6n2tj\""
Jul 08 06:55:05.911122 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.911080    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4f688\""
Jul 08 06:55:05.911268 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.911208    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 06:55:05.911268 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.911243    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 06:55:05.911371 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.911302    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 06:55:05.952852 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.952825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e96bc4c3-a102-47da-8a20-fa8c17a05c83-config-volume\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.952968 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.952857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96bc4c3-a102-47da-8a20-fa8c17a05c83-tmp-dir\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.952968 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.952890    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4077e488-ca01-4396-9f45-cebd5fbdcef1-ca-trust-extracted\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.952968 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.952920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dqm6\" (UniqueName: \"kubernetes.io/projected/f46102e8-813f-45bd-bcad-cac9e30b7d70-kube-api-access-5dqm6\") pod \"ingress-canary-fm69n\" (UID: \"f46102e8-813f-45bd-bcad-cac9e30b7d70\") " pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.952968 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.952946    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-trusted-ca\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.953858 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.953832    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e96bc4c3-a102-47da-8a20-fa8c17a05c83-config-volume\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.953967 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.953074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd8rk\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-kube-api-access-pd8rk\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.953967 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.953930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.953967 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.953953    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96bc4c3-a102-47da-8a20-fa8c17a05c83-tmp-dir\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.954115 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.953968    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-certificates\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.954115 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.953878    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4077e488-ca01-4396-9f45-cebd5fbdcef1-ca-trust-extracted\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.954115 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.954004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-bound-sa-token\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.954115 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.954096    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:05.954115 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.954110    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bd9657ff9-lv66b: secret "image-registry-tls" not found
Jul 08 06:55:05.954336 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.954168    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls podName:4077e488-ca01-4396-9f45-cebd5fbdcef1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:06.454150617 +0000 UTC m=+34.053440304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls") pod "image-registry-7bd9657ff9-lv66b" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1") : secret "image-registry-tls" not found
Jul 08 06:55:05.954442 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.954423    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.954583 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.954568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvxc\" (UniqueName: \"kubernetes.io/projected/e96bc4c3-a102-47da-8a20-fa8c17a05c83-kube-api-access-srvxc\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:05.954646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.954613    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f46102e8-813f-45bd-bcad-cac9e30b7d70-cert\") pod \"ingress-canary-fm69n\" (UID: \"f46102e8-813f-45bd-bcad-cac9e30b7d70\") " pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.954789 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.954775    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-image-registry-private-configuration\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.954852 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.954825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-installation-pull-secrets\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.955428 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.955407    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-certificates\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.955518 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.955452    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:05.955637 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:05.955615    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls podName:e96bc4c3-a102-47da-8a20-fa8c17a05c83 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:06.455599513 +0000 UTC m=+34.054889209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls") pod "dns-default-g468f" (UID: "e96bc4c3-a102-47da-8a20-fa8c17a05c83") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:05.957356 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.957335    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-trusted-ca\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.959518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.959497    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-installation-pull-secrets\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.959633 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.959602    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-image-registry-private-configuration\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.959711 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.959636    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f46102e8-813f-45bd-bcad-cac9e30b7d70-cert\") pod \"ingress-canary-fm69n\" (UID: \"f46102e8-813f-45bd-bcad-cac9e30b7d70\") " pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.970550 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.970512    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dqm6\" (UniqueName: \"kubernetes.io/projected/f46102e8-813f-45bd-bcad-cac9e30b7d70-kube-api-access-5dqm6\") pod \"ingress-canary-fm69n\" (UID: \"f46102e8-813f-45bd-bcad-cac9e30b7d70\") " pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:05.972056 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.972036    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-bound-sa-token\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.972570 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.972548    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd8rk\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-kube-api-access-pd8rk\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:05.973013 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:05.972985    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvxc\" (UniqueName: \"kubernetes.io/projected/e96bc4c3-a102-47da-8a20-fa8c17a05c83-kube-api-access-srvxc\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:06.038347 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:06.038310    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fm69n"
Jul 08 06:55:06.206048 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:06.206022    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fm69n"]
Jul 08 06:55:06.209426 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:06.209399    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46102e8_813f_45bd_bcad_cac9e30b7d70.slice/crio-c1b92a32777e40be29c3e6e7960103aab2a9daa7af27827e80f7b4f0d3b48b50 WatchSource:0}: Error finding container c1b92a32777e40be29c3e6e7960103aab2a9daa7af27827e80f7b4f0d3b48b50: Status 404 returned error can't find the container with id c1b92a32777e40be29c3e6e7960103aab2a9daa7af27827e80f7b4f0d3b48b50
Jul 08 06:55:06.459422 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:06.459338    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:06.459584 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:06.459425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:06.459584 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:06.459520    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:06.459584 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:06.459521    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:06.459805 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:06.459615    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls podName:e96bc4c3-a102-47da-8a20-fa8c17a05c83 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:07.459593697 +0000 UTC m=+35.058883394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls") pod "dns-default-g468f" (UID: "e96bc4c3-a102-47da-8a20-fa8c17a05c83") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:06.459805 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:06.459532    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bd9657ff9-lv66b: secret "image-registry-tls" not found
Jul 08 06:55:06.459805 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:06.459731    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls podName:4077e488-ca01-4396-9f45-cebd5fbdcef1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:07.459709381 +0000 UTC m=+35.058999083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls") pod "image-registry-7bd9657ff9-lv66b" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1") : secret "image-registry-tls" not found
Jul 08 06:55:07.142847 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:07.142577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fm69n" event={"ID":"f46102e8-813f-45bd-bcad-cac9e30b7d70","Type":"ContainerStarted","Data":"c1b92a32777e40be29c3e6e7960103aab2a9daa7af27827e80f7b4f0d3b48b50"}
Jul 08 06:55:07.468948 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:07.468855    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:07.468948 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:07.468909    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:07.469160 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:07.469033    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:07.469160 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:07.469049    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:07.469160 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:07.469057    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bd9657ff9-lv66b: secret "image-registry-tls" not found
Jul 08 06:55:07.469160 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:07.469148    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls podName:e96bc4c3-a102-47da-8a20-fa8c17a05c83 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:09.469099121 +0000 UTC m=+37.068388805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls") pod "dns-default-g468f" (UID: "e96bc4c3-a102-47da-8a20-fa8c17a05c83") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:07.469368 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:07.469175    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls podName:4077e488-ca01-4396-9f45-cebd5fbdcef1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:09.469160371 +0000 UTC m=+37.068450053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls") pod "image-registry-7bd9657ff9-lv66b" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1") : secret "image-registry-tls" not found
Jul 08 06:55:09.484425 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:09.484397    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:09.484711 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:09.484454    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:09.484711 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:09.484581    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:09.484711 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:09.484595    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:09.484711 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:09.484606    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bd9657ff9-lv66b: secret "image-registry-tls" not found
Jul 08 06:55:09.484711 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:09.484648    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls podName:e96bc4c3-a102-47da-8a20-fa8c17a05c83 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:13.484634196 +0000 UTC m=+41.083923879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls") pod "dns-default-g468f" (UID: "e96bc4c3-a102-47da-8a20-fa8c17a05c83") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:09.484711 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:09.484662    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls podName:4077e488-ca01-4396-9f45-cebd5fbdcef1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:13.484655876 +0000 UTC m=+41.083945557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls") pod "image-registry-7bd9657ff9-lv66b" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1") : secret "image-registry-tls" not found
Jul 08 06:55:10.152072 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:10.152041    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ee5adca-42ed-4f10-b33c-c05a981388f9" containerID="15720cbb9aad2bc243dc8df24c0ae3ac1b6e35ffcd50a17dacc8a234f122216f" exitCode=0
Jul 08 06:55:10.152308 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:10.152123    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerDied","Data":"15720cbb9aad2bc243dc8df24c0ae3ac1b6e35ffcd50a17dacc8a234f122216f"}
Jul 08 06:55:10.153427 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:10.153406    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fm69n" event={"ID":"f46102e8-813f-45bd-bcad-cac9e30b7d70","Type":"ContainerStarted","Data":"0d8f7785bb2ef8da9eef9caf76bc0fb35f5a2cffead31fe9db7cd92a7dcef674"}
Jul 08 06:55:10.191295 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:10.191248    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fm69n" podStartSLOduration=1.9454138859999999 podStartE2EDuration="5.191232342s" totalImagesPullingTime="3.245818456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:10.19042754 +0000 UTC m=+37.789717244" watchObservedRunningTime="2026-07-08 06:55:10.191232342 +0000 UTC m=+37.790522045"
Jul 08 06:55:11.158566 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:11.158520    2571 generic.go:356] "Generic (PLEG): container finished" podID="7ee5adca-42ed-4f10-b33c-c05a981388f9" containerID="038ece9db6fda7a10022c4034cb97b0483d910070dce10e714dcaafd5885c504" exitCode=0
Jul 08 06:55:11.158959 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:11.158600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerDied","Data":"038ece9db6fda7a10022c4034cb97b0483d910070dce10e714dcaafd5885c504"}
Jul 08 06:55:12.164704 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:12.164654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" event={"ID":"7ee5adca-42ed-4f10-b33c-c05a981388f9","Type":"ContainerStarted","Data":"a4ee13035ba554c54d308ea1ea01bf2beca98169de35613bf1288a3337f39634"}
Jul 08 06:55:12.191076 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:12.191029    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xt7ss" podStartSLOduration=7.504840939 podStartE2EDuration="39.191012586s" totalImagesPullingTime="31.686171647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:12.188648997 +0000 UTC m=+39.787938700" watchObservedRunningTime="2026-07-08 06:55:12.191012586 +0000 UTC m=+39.790302290"
Jul 08 06:55:13.515211 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:13.515172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:13.515577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:13.515278    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:13.515577 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:13.515347    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:13.515577 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:13.515373    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:13.515577 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:13.515385    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bd9657ff9-lv66b: secret "image-registry-tls" not found
Jul 08 06:55:13.515577 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:13.515425    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls podName:e96bc4c3-a102-47da-8a20-fa8c17a05c83 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:21.515399584 +0000 UTC m=+49.114689285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls") pod "dns-default-g468f" (UID: "e96bc4c3-a102-47da-8a20-fa8c17a05c83") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:13.515577 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:13.515440    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls podName:4077e488-ca01-4396-9f45-cebd5fbdcef1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:21.515434139 +0000 UTC m=+49.114723823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls") pod "image-registry-7bd9657ff9-lv66b" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1") : secret "image-registry-tls" not found
Jul 08 06:55:20.700502 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.700468    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"]
Jul 08 06:55:20.705800 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.703155    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.705800 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.704790    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-g2pqf"]
Jul 08 06:55:20.707360 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.707341    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"]
Jul 08 06:55:20.707483 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.707467    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.707599 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.707472    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hhkzg\""
Jul 08 06:55:20.708321 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.708303    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 08 06:55:20.708566 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.708552    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 08 06:55:20.709227 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.709208    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.709986 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.709970    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 08 06:55:20.710398 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.710384    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 08 06:55:20.710724 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.710707    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 08 06:55:20.711204 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.711183    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nlkbh\""
Jul 08 06:55:20.711297 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.711282    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 06:55:20.711489 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.711474    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 06:55:20.711561 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.711548    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 06:55:20.713910 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.713891    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 08 06:55:20.713992 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.713913    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 08 06:55:20.714820 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.714800    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-67s7k\""
Jul 08 06:55:20.719584 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.719553    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 06:55:20.720250 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.720233    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 08 06:55:20.721211 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.721167    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 08 06:55:20.721281 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.721203    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"]
Jul 08 06:55:20.740176 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.740148    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"]
Jul 08 06:55:20.740955 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.740929    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-g2pqf"]
Jul 08 06:55:20.867750 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867714    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c5f560-1820-422f-823b-7c90342c0793-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.867750 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00afd1f2-a8a8-4192-861f-733bc074b996-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.867953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00afd1f2-a8a8-4192-861f-733bc074b996-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.867953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867841    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhvr5\" (UniqueName: \"kubernetes.io/projected/00afd1f2-a8a8-4192-861f-733bc074b996-kube-api-access-lhvr5\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.867953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867884    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.867953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867909    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00afd1f2-a8a8-4192-861f-733bc074b996-serving-cert\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.867953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867936    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00afd1f2-a8a8-4192-861f-733bc074b996-snapshots\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.868124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.867955    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.868124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.868011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nd4\" (UniqueName: \"kubernetes.io/projected/09c5f560-1820-422f-823b-7c90342c0793-kube-api-access-n6nd4\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.868124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.868032    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c5f560-1820-422f-823b-7c90342c0793-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.868124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.868054    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00afd1f2-a8a8-4192-861f-733bc074b996-tmp\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.868124 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.868104    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8zw\" (UniqueName: \"kubernetes.io/projected/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-kube-api-access-5b8zw\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.968460 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968372    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.968460 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968407    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00afd1f2-a8a8-4192-861f-733bc074b996-serving-cert\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.968460 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00afd1f2-a8a8-4192-861f-733bc074b996-snapshots\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.968460 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968445    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968501    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nd4\" (UniqueName: \"kubernetes.io/projected/09c5f560-1820-422f-823b-7c90342c0793-kube-api-access-n6nd4\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968528    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c5f560-1820-422f-823b-7c90342c0793-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968552    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00afd1f2-a8a8-4192-861f-733bc074b996-tmp\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8zw\" (UniqueName: \"kubernetes.io/projected/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-kube-api-access-5b8zw\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c5f560-1820-422f-823b-7c90342c0793-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968766    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00afd1f2-a8a8-4192-861f-733bc074b996-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00afd1f2-a8a8-4192-861f-733bc074b996-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.968823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.968813    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhvr5\" (UniqueName: \"kubernetes.io/projected/00afd1f2-a8a8-4192-861f-733bc074b996-kube-api-access-lhvr5\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.969227 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.969202    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00afd1f2-a8a8-4192-861f-733bc074b996-snapshots\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.969280 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:20.968654    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:20.969327 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:20.969304    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls podName:f5bec0a8-4daa-4dc8-ad04-9b399c5f5826 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:21.469282824 +0000 UTC m=+49.068572505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qvvg6" (UID: "f5bec0a8-4daa-4dc8-ad04-9b399c5f5826") : secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:20.969327 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.969315    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:20.969422 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.969393    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c5f560-1820-422f-823b-7c90342c0793-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.969515 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.969496    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00afd1f2-a8a8-4192-861f-733bc074b996-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.969729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.969681    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00afd1f2-a8a8-4192-861f-733bc074b996-tmp\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.969947 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.969931    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00afd1f2-a8a8-4192-861f-733bc074b996-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.971804 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.971780    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c5f560-1820-422f-823b-7c90342c0793-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.971890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.971824    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00afd1f2-a8a8-4192-861f-733bc074b996-serving-cert\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.977560 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.977534    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nd4\" (UniqueName: \"kubernetes.io/projected/09c5f560-1820-422f-823b-7c90342c0793-kube-api-access-n6nd4\") pod \"kube-storage-version-migrator-operator-846bf668f7-hwn2m\" (UID: \"09c5f560-1820-422f-823b-7c90342c0793\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:20.978334 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.978308    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhvr5\" (UniqueName: \"kubernetes.io/projected/00afd1f2-a8a8-4192-861f-733bc074b996-kube-api-access-lhvr5\") pod \"insights-operator-5c847dcfc4-g2pqf\" (UID: \"00afd1f2-a8a8-4192-861f-733bc074b996\") " pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:20.978600 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:20.978585    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8zw\" (UniqueName: \"kubernetes.io/projected/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-kube-api-access-5b8zw\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:21.017731 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.017675    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"
Jul 08 06:55:21.024378 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.024351    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf"
Jul 08 06:55:21.161092 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.161052    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m"]
Jul 08 06:55:21.163935 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:21.163901    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c5f560_1820_422f_823b_7c90342c0793.slice/crio-a599a8b558a9416032b147b0d37f0a38ad63707abd7a634a41f2b2f26e3bea62 WatchSource:0}: Error finding container a599a8b558a9416032b147b0d37f0a38ad63707abd7a634a41f2b2f26e3bea62: Status 404 returned error can't find the container with id a599a8b558a9416032b147b0d37f0a38ad63707abd7a634a41f2b2f26e3bea62
Jul 08 06:55:21.178094 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.178071    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-g2pqf"]
Jul 08 06:55:21.180680 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:21.180657    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00afd1f2_a8a8_4192_861f_733bc074b996.slice/crio-65457c9288a2acf7a35bebba653838c6b7ef0df96525b73d3be41b38b423b5f6 WatchSource:0}: Error finding container 65457c9288a2acf7a35bebba653838c6b7ef0df96525b73d3be41b38b423b5f6: Status 404 returned error can't find the container with id 65457c9288a2acf7a35bebba653838c6b7ef0df96525b73d3be41b38b423b5f6
Jul 08 06:55:21.185165 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.185140    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf" event={"ID":"00afd1f2-a8a8-4192-861f-733bc074b996","Type":"ContainerStarted","Data":"65457c9288a2acf7a35bebba653838c6b7ef0df96525b73d3be41b38b423b5f6"}
Jul 08 06:55:21.186157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.186135    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m" event={"ID":"09c5f560-1820-422f-823b-7c90342c0793","Type":"ContainerStarted","Data":"a599a8b558a9416032b147b0d37f0a38ad63707abd7a634a41f2b2f26e3bea62"}
Jul 08 06:55:21.473350 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.473317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:21.473526 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.473482    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:21.473568 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.473551    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls podName:f5bec0a8-4daa-4dc8-ad04-9b399c5f5826 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:22.473533057 +0000 UTC m=+50.072822740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qvvg6" (UID: "f5bec0a8-4daa-4dc8-ad04-9b399c5f5826") : secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:21.574071 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.574019    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:21.574071 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.574070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:21.574362 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.574178    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 06:55:21.574362 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.574181    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 06:55:21.574362 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.574199    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bd9657ff9-lv66b: secret "image-registry-tls" not found
Jul 08 06:55:21.574362 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.574247    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls podName:4077e488-ca01-4396-9f45-cebd5fbdcef1 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:37.574227098 +0000 UTC m=+65.173516787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls") pod "image-registry-7bd9657ff9-lv66b" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1") : secret "image-registry-tls" not found
Jul 08 06:55:21.574362 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:21.574262    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls podName:e96bc4c3-a102-47da-8a20-fa8c17a05c83 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:37.574254152 +0000 UTC m=+65.173543840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls") pod "dns-default-g468f" (UID: "e96bc4c3-a102-47da-8a20-fa8c17a05c83") : secret "dns-default-metrics-tls" not found
Jul 08 06:55:21.905309 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.905273    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"]
Jul 08 06:55:21.907238 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.907209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:21.910942 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.910918    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tjrx4\""
Jul 08 06:55:21.911079 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.910956    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 08 06:55:21.916305 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.916277    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 08 06:55:21.976133 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:21.976027    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"]
Jul 08 06:55:22.078949 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.078743    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:22.078949 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.078815    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:22.180252 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.180165    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:22.180404 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.180270    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:22.180404 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:22.180331    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 06:55:22.180404 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:22.180398    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert podName:ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:22.680381628 +0000 UTC m=+50.279671309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qxs7d" (UID: "ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8") : secret "networking-console-plugin-cert" not found
Jul 08 06:55:22.181577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.181549    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:22.483766 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.483085    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:22.483766 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:22.483316    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:22.483766 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:22.483374    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls podName:f5bec0a8-4daa-4dc8-ad04-9b399c5f5826 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:24.483359282 +0000 UTC m=+52.082648981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qvvg6" (UID: "f5bec0a8-4daa-4dc8-ad04-9b399c5f5826") : secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:22.684546 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:22.684504    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:22.684739 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:22.684660    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 06:55:22.684821 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:22.684765    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert podName:ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:23.684743672 +0000 UTC m=+51.284033355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qxs7d" (UID: "ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8") : secret "networking-console-plugin-cert" not found
Jul 08 06:55:23.694005 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:23.693912    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:23.694358 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:23.694094    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 06:55:23.694358 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:23.694220    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert podName:ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:25.694197584 +0000 UTC m=+53.293487271 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qxs7d" (UID: "ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8") : secret "networking-console-plugin-cert" not found
Jul 08 06:55:24.196023 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.195977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m" event={"ID":"09c5f560-1820-422f-823b-7c90342c0793","Type":"ContainerStarted","Data":"eb95962fc983aa3462ef304c513233aabd83f3e6374416996b43a49ece09b0bd"}
Jul 08 06:55:24.197717 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.197667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf" event={"ID":"00afd1f2-a8a8-4192-861f-733bc074b996","Type":"ContainerStarted","Data":"3863493dbc3a528cf4c41f01e1a826cbbb857c129a1f93f89862009683e6b21a"}
Jul 08 06:55:24.222562 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.222506    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m" podStartSLOduration=1.9538413989999999 podStartE2EDuration="4.222490313s" totalImagesPullingTime="2.268648914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:24.222054473 +0000 UTC m=+51.821344178" watchObservedRunningTime="2026-07-08 06:55:24.222490313 +0000 UTC m=+51.821780015"
Jul 08 06:55:24.501294 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.501209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:24.501432 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:24.501355    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:24.501432 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:24.501422    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls podName:f5bec0a8-4daa-4dc8-ad04-9b399c5f5826 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:28.501407167 +0000 UTC m=+56.100696848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qvvg6" (UID: "f5bec0a8-4daa-4dc8-ad04-9b399c5f5826") : secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:24.689416 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.689318    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf" podStartSLOduration=2.440698611 podStartE2EDuration="4.689294751s" totalImagesPullingTime="2.24859614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:24.263100053 +0000 UTC m=+51.862389770" watchObservedRunningTime="2026-07-08 06:55:24.689294751 +0000 UTC m=+52.288584455"
Jul 08 06:55:24.690355 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.690329    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-glfl6"]
Jul 08 06:55:24.693033 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.693014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.697710 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.697509    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h6mll\""
Jul 08 06:55:24.697710 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.697625    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 06:55:24.698181 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.698155    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 06:55:24.720085 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.720060    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-glfl6"]
Jul 08 06:55:24.807518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.807437    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a81c997-4343-4bda-a495-4753336ff8bb-data-volume\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.807518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.807483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.807722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.807557    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgzpp\" (UniqueName: \"kubernetes.io/projected/9a81c997-4343-4bda-a495-4753336ff8bb-kube-api-access-wgzpp\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.807722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.807594    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a81c997-4343-4bda-a495-4753336ff8bb-crio-socket\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.807722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.807659    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.908934 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.908891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a81c997-4343-4bda-a495-4753336ff8bb-crio-socket\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.909103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.908982    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.909103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.909009    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a81c997-4343-4bda-a495-4753336ff8bb-crio-socket\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.909103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.909059    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a81c997-4343-4bda-a495-4753336ff8bb-data-volume\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.909103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.909078    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.909103 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:24.909091    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:55:24.909336 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.909124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgzpp\" (UniqueName: \"kubernetes.io/projected/9a81c997-4343-4bda-a495-4753336ff8bb-kube-api-access-wgzpp\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.909336 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:24.909157    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:25.409138068 +0000 UTC m=+53.008427766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : configmap "kube-rbac-proxy" not found
Jul 08 06:55:24.909336 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:24.909260    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:24.909336 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:24.909317    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:25.409300466 +0000 UTC m=+53.008590162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:24.909547 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.909492    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a81c997-4343-4bda-a495-4753336ff8bb-data-volume\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:24.918024 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:24.917989    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgzpp\" (UniqueName: \"kubernetes.io/projected/9a81c997-4343-4bda-a495-4753336ff8bb-kube-api-access-wgzpp\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:25.413327 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:25.413297    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:25.413490 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:25.413360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:25.413490 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:25.413423    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:55:25.413490 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:25.413487    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:26.413471013 +0000 UTC m=+54.012760697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : configmap "kube-rbac-proxy" not found
Jul 08 06:55:25.413490 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:25.413487    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:25.413653 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:25.413529    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:26.413518165 +0000 UTC m=+54.012807846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:25.716718 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:25.716603    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:25.717163 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:25.716799    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 06:55:25.717163 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:25.716885    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert podName:ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:29.716863886 +0000 UTC m=+57.316153572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qxs7d" (UID: "ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8") : secret "networking-console-plugin-cert" not found
Jul 08 06:55:26.423743 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:26.423679    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:26.423993 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:26.423771    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:55:26.423993 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:26.423786    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:26.423993 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:26.423838    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:28.423818784 +0000 UTC m=+56.023108468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : configmap "kube-rbac-proxy" not found
Jul 08 06:55:26.423993 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:26.423945    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:26.424130 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:26.423998    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:28.423983907 +0000 UTC m=+56.023273600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:27.436955 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:27.436927    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b8bd9_e1fd811b-ecd4-4720-b927-556e91ace214/dns-node-resolver/0.log"
Jul 08 06:55:28.440646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:28.440608    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:28.441112 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:28.440698    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:28.441112 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:28.440750    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:55:28.441112 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:28.440838    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:28.441112 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:28.440846    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:32.440823245 +0000 UTC m=+60.040112946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : configmap "kube-rbac-proxy" not found
Jul 08 06:55:28.441112 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:28.440896    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:32.440884246 +0000 UTC m=+60.040173930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:28.541639 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:28.541602    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:28.541840 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:28.541751    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:28.541840 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:28.541806    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls podName:f5bec0a8-4daa-4dc8-ad04-9b399c5f5826 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:36.541791851 +0000 UTC m=+64.141081532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qvvg6" (UID: "f5bec0a8-4daa-4dc8-ad04-9b399c5f5826") : secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:28.636931 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:28.636903    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kkdj4_65032d2e-e6b0-4469-9921-3273ef4e5f75/node-ca/0.log"
Jul 08 06:55:29.040916 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:29.040886    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fm69n_f46102e8-813f-45bd-bcad-cac9e30b7d70/serve-healthcheck-canary/0.log"
Jul 08 06:55:29.750702 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:29.750650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:29.751058 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:29.750806    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 06:55:29.751058 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:29.750882    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert podName:ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:37.750864691 +0000 UTC m=+65.350154379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qxs7d" (UID: "ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8") : secret "networking-console-plugin-cert" not found
Jul 08 06:55:30.038740 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:30.038636    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hwn2m_09c5f560-1820-422f-823b-7c90342c0793/kube-storage-version-migrator-operator/0.log"
Jul 08 06:55:32.140031 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:32.140006    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n5zzg"
Jul 08 06:55:32.472457 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:32.472368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:32.472590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:32.472486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:32.472590 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:32.472549    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:32.472590 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:32.472575    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 06:55:32.472722 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:32.472632    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:40.472610165 +0000 UTC m=+68.071899850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:32.472722 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:32.472653    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:40.472643389 +0000 UTC m=+68.071933098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : configmap "kube-rbac-proxy" not found
Jul 08 06:55:36.608529 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:36.608485    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:36.609017 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:36.608640    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:36.609017 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:36.608737    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls podName:f5bec0a8-4daa-4dc8-ad04-9b399c5f5826 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:52.608720405 +0000 UTC m=+80.208010086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qvvg6" (UID: "f5bec0a8-4daa-4dc8-ad04-9b399c5f5826") : secret "cluster-monitoring-operator-tls" not found
Jul 08 06:55:37.619567 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.619525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:37.619567 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.619566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:37.620073 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.619611    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:55:37.620073 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.619658    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:37.622050 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.622019    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e96bc4c3-a102-47da-8a20-fa8c17a05c83-metrics-tls\") pod \"dns-default-g468f\" (UID: \"e96bc4c3-a102-47da-8a20-fa8c17a05c83\") " pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:37.622257 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.622235    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"image-registry-7bd9657ff9-lv66b\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") " pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:37.622300 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.622252    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 06:55:37.622300 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.622250    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 06:55:37.630178 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:37.630160    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 06:55:37.630257 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:37.630216    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs podName:2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624 nodeName:}" failed. No retries permitted until 2026-07-08 06:56:41.630199891 +0000 UTC m=+129.229489575 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs") pod "network-metrics-daemon-vtskv" (UID: "2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624") : secret "metrics-daemon-secret" not found
Jul 08 06:55:37.632052 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.632033    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/486b3034-61ca-49ad-9061-bf7c9369447c-original-pull-secret\") pod \"global-pull-secret-syncer-6qqpr\" (UID: \"486b3034-61ca-49ad-9061-bf7c9369447c\") " pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:37.720786 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.720756    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:37.723940 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.723922    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 06:55:37.733656 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.733640    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 06:55:37.744326 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.744305    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncgvc\" (UniqueName: \"kubernetes.io/projected/95e16ab3-b354-4c20-a5fe-d8039ed13111-kube-api-access-ncgvc\") pod \"network-check-target-lb9c2\" (UID: \"95e16ab3-b354-4c20-a5fe-d8039ed13111\") " pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:37.747093 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.747076    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qqpr"
Jul 08 06:55:37.812853 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.812815    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q8mxm\""
Jul 08 06:55:37.821869 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.821062    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:37.821869 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.821434    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:37.821869 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:37.821562    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 08 06:55:37.821869 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:37.821617    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert podName:ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8 nodeName:}" failed. No retries permitted until 2026-07-08 06:55:53.821599012 +0000 UTC m=+81.420888709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-qxs7d" (UID: "ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8") : secret "networking-console-plugin-cert" not found
Jul 08 06:55:37.831619 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.831597    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6zr5\""
Jul 08 06:55:37.839946 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.839878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:37.906372 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.906175    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6qqpr"]
Jul 08 06:55:37.910073 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:37.910045    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486b3034_61ca_49ad_9061_bf7c9369447c.slice/crio-e69e654573052724d0bbcdcbf4900a8fd966e45381a8e259822c83a737fb47b6 WatchSource:0}: Error finding container e69e654573052724d0bbcdcbf4900a8fd966e45381a8e259822c83a737fb47b6: Status 404 returned error can't find the container with id e69e654573052724d0bbcdcbf4900a8fd966e45381a8e259822c83a737fb47b6
Jul 08 06:55:37.995360 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:37.995335    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bd9657ff9-lv66b"]
Jul 08 06:55:37.997680 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:37.997655    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4077e488_ca01_4396_9f45_cebd5fbdcef1.slice/crio-2c8bc61e6cfcb9af9326c47ce4502a3ed6b54de1cf1084e6f3380e4ce2015fa5 WatchSource:0}: Error finding container 2c8bc61e6cfcb9af9326c47ce4502a3ed6b54de1cf1084e6f3380e4ce2015fa5: Status 404 returned error can't find the container with id 2c8bc61e6cfcb9af9326c47ce4502a3ed6b54de1cf1084e6f3380e4ce2015fa5
Jul 08 06:55:38.002265 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.002243    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g468f"]
Jul 08 06:55:38.004872 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:38.004854    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96bc4c3_a102_47da_8a20_fa8c17a05c83.slice/crio-b0f38503dbb407dc0cdc62ee830a007f7539b45812be77975ba061530ab393b4 WatchSource:0}: Error finding container b0f38503dbb407dc0cdc62ee830a007f7539b45812be77975ba061530ab393b4: Status 404 returned error can't find the container with id b0f38503dbb407dc0cdc62ee830a007f7539b45812be77975ba061530ab393b4
Jul 08 06:55:38.021362 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.021341    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6n2tj\""
Jul 08 06:55:38.028727 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.028705    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:38.170604 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.170566    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lb9c2"]
Jul 08 06:55:38.174249 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:38.174226    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e16ab3_b354_4c20_a5fe_d8039ed13111.slice/crio-e1f9d9cf7f4b65aae304a10fa82d70da37aefc9c7acb8112061fbbf93a6edb47 WatchSource:0}: Error finding container e1f9d9cf7f4b65aae304a10fa82d70da37aefc9c7acb8112061fbbf93a6edb47: Status 404 returned error can't find the container with id e1f9d9cf7f4b65aae304a10fa82d70da37aefc9c7acb8112061fbbf93a6edb47
Jul 08 06:55:38.231550 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.231518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6qqpr" event={"ID":"486b3034-61ca-49ad-9061-bf7c9369447c","Type":"ContainerStarted","Data":"e69e654573052724d0bbcdcbf4900a8fd966e45381a8e259822c83a737fb47b6"}
Jul 08 06:55:38.232633 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.232610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lb9c2" event={"ID":"95e16ab3-b354-4c20-a5fe-d8039ed13111","Type":"ContainerStarted","Data":"e1f9d9cf7f4b65aae304a10fa82d70da37aefc9c7acb8112061fbbf93a6edb47"}
Jul 08 06:55:38.233836 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.233806    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g468f" event={"ID":"e96bc4c3-a102-47da-8a20-fa8c17a05c83","Type":"ContainerStarted","Data":"b0f38503dbb407dc0cdc62ee830a007f7539b45812be77975ba061530ab393b4"}
Jul 08 06:55:38.235361 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.235340    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" event={"ID":"4077e488-ca01-4396-9f45-cebd5fbdcef1","Type":"ContainerStarted","Data":"853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb"}
Jul 08 06:55:38.235454 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.235379    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" event={"ID":"4077e488-ca01-4396-9f45-cebd5fbdcef1","Type":"ContainerStarted","Data":"2c8bc61e6cfcb9af9326c47ce4502a3ed6b54de1cf1084e6f3380e4ce2015fa5"}
Jul 08 06:55:38.235511 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.235474    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:38.260448 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:38.260402    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" podStartSLOduration=65.260386637 podStartE2EDuration="1m5.260386637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:38.260089163 +0000 UTC m=+65.859378866" watchObservedRunningTime="2026-07-08 06:55:38.260386637 +0000 UTC m=+65.859676339"
Jul 08 06:55:40.547717 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:40.547630    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:40.547717 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:40.547708    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:40.548126 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:40.547852    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:40.548126 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:55:40.547931    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls podName:9a81c997-4343-4bda-a495-4753336ff8bb nodeName:}" failed. No retries permitted until 2026-07-08 06:55:56.54791237 +0000 UTC m=+84.147202071 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-glfl6" (UID: "9a81c997-4343-4bda-a495-4753336ff8bb") : secret "insights-runtime-extractor-tls" not found
Jul 08 06:55:40.548317 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:40.548290    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a81c997-4343-4bda-a495-4753336ff8bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:41.245749 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:41.245712    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g468f" event={"ID":"e96bc4c3-a102-47da-8a20-fa8c17a05c83","Type":"ContainerStarted","Data":"9f1a3693870ca39bb37950fcd319bcb4202c47dec495ace1640621b13f3ac8b8"}
Jul 08 06:55:43.252358 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.252317    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lb9c2" event={"ID":"95e16ab3-b354-4c20-a5fe-d8039ed13111","Type":"ContainerStarted","Data":"339dab0151a49f205e50303b36c8903a986425ef2057071d1b3ef696adc89b77"}
Jul 08 06:55:43.252841 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.252417    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:55:43.254165 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.254137    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g468f" event={"ID":"e96bc4c3-a102-47da-8a20-fa8c17a05c83","Type":"ContainerStarted","Data":"9d13e0dbfd5e31eaf4ac3b78f466b8e4aeb4ba98568c48d1d42c453c9b137845"}
Jul 08 06:55:43.254291 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.254238    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:43.255495 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.255473    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6qqpr" event={"ID":"486b3034-61ca-49ad-9061-bf7c9369447c","Type":"ContainerStarted","Data":"a075da22e3ae21a25cb384d1d689e94c2a477229785cf870a65d605f4d958289"}
Jul 08 06:55:43.269509 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.269465    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lb9c2" podStartSLOduration=65.657362167 podStartE2EDuration="1m10.269451865s" totalImagesPullingTime="4.612089698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:43.268726866 +0000 UTC m=+70.868016565" watchObservedRunningTime="2026-07-08 06:55:43.269451865 +0000 UTC m=+70.868741574"
Jul 08 06:55:43.285072 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.285035    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6qqpr" podStartSLOduration=65.401209013 podStartE2EDuration="1m10.285023653s" totalImagesPullingTime="4.88381464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:43.283816439 +0000 UTC m=+70.883106120" watchObservedRunningTime="2026-07-08 06:55:43.285023653 +0000 UTC m=+70.884313356"
Jul 08 06:55:43.302408 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:43.302371    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g468f" podStartSLOduration=36.085447948 podStartE2EDuration="38.302359592s" totalImagesPullingTime="2.216911644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:43.302261254 +0000 UTC m=+70.901550960" watchObservedRunningTime="2026-07-08 06:55:43.302359592 +0000 UTC m=+70.901649295"
Jul 08 06:55:50.262707 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:50.262654    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g468f"
Jul 08 06:55:52.640156 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:52.640103    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:52.642650 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:52.642630    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f5bec0a8-4daa-4dc8-ad04-9b399c5f5826-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qvvg6\" (UID: \"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:52.831871 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:52.831839    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-67s7k\""
Jul 08 06:55:52.839988 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:52.839968    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"
Jul 08 06:55:52.974449 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:52.974406    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6"]
Jul 08 06:55:52.976949 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:52.976921    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5bec0a8_4daa_4dc8_ad04_9b399c5f5826.slice/crio-672067a0cbfa97a59b656345a021c2393807f9c5f5899d61a1a3bfa68c986d40 WatchSource:0}: Error finding container 672067a0cbfa97a59b656345a021c2393807f9c5f5899d61a1a3bfa68c986d40: Status 404 returned error can't find the container with id 672067a0cbfa97a59b656345a021c2393807f9c5f5899d61a1a3bfa68c986d40
Jul 08 06:55:53.127608 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.127570    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"]
Jul 08 06:55:53.129983 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.129961    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.132980 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.132948    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 08 06:55:53.132980 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.132955    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 08 06:55:53.132980 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.132982    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 08 06:55:53.133878 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.133853    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 08 06:55:53.134015 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.133997    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lfm9x\""
Jul 08 06:55:53.148870 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.148846    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"]
Jul 08 06:55:53.256288 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.256067    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43230123-d998-4f29-9dbe-104648246888-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-849686c7c8-lthjp\" (UID: \"43230123-d998-4f29-9dbe-104648246888\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.256288 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.256142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv45g\" (UniqueName: \"kubernetes.io/projected/43230123-d998-4f29-9dbe-104648246888-kube-api-access-fv45g\") pod \"managed-serviceaccount-addon-agent-849686c7c8-lthjp\" (UID: \"43230123-d998-4f29-9dbe-104648246888\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.293057 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.293023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6" event={"ID":"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826","Type":"ContainerStarted","Data":"672067a0cbfa97a59b656345a021c2393807f9c5f5899d61a1a3bfa68c986d40"}
Jul 08 06:55:53.357635 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.357588    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43230123-d998-4f29-9dbe-104648246888-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-849686c7c8-lthjp\" (UID: \"43230123-d998-4f29-9dbe-104648246888\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.357823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.357657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv45g\" (UniqueName: \"kubernetes.io/projected/43230123-d998-4f29-9dbe-104648246888-kube-api-access-fv45g\") pod \"managed-serviceaccount-addon-agent-849686c7c8-lthjp\" (UID: \"43230123-d998-4f29-9dbe-104648246888\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.360480 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.360458    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43230123-d998-4f29-9dbe-104648246888-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-849686c7c8-lthjp\" (UID: \"43230123-d998-4f29-9dbe-104648246888\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.373960 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.373933    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv45g\" (UniqueName: \"kubernetes.io/projected/43230123-d998-4f29-9dbe-104648246888-kube-api-access-fv45g\") pod \"managed-serviceaccount-addon-agent-849686c7c8-lthjp\" (UID: \"43230123-d998-4f29-9dbe-104648246888\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.452097 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.452035    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"
Jul 08 06:55:53.598849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.598812    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp"]
Jul 08 06:55:53.603877 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:53.603850    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43230123_d998_4f29_9dbe_104648246888.slice/crio-754c93f53adba888a3d167c87bc3fafc65ecf1fc0600a805458144c2ace5553d WatchSource:0}: Error finding container 754c93f53adba888a3d167c87bc3fafc65ecf1fc0600a805458144c2ace5553d: Status 404 returned error can't find the container with id 754c93f53adba888a3d167c87bc3fafc65ecf1fc0600a805458144c2ace5553d
Jul 08 06:55:53.862253 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.862213    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:53.865104 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:53.865076    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qxs7d\" (UID: \"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:54.032905 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.032793    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tjrx4\""
Jul 08 06:55:54.036048 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.036025    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"
Jul 08 06:55:54.297473 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.297389    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp" event={"ID":"43230123-d998-4f29-9dbe-104648246888","Type":"ContainerStarted","Data":"754c93f53adba888a3d167c87bc3fafc65ecf1fc0600a805458144c2ace5553d"}
Jul 08 06:55:54.527008 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.526975    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d"]
Jul 08 06:55:54.530462 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:54.530432    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae13f9c0_58e6_44a3_a5ba_ddcd0174c5a8.slice/crio-7431da64a25947f7b63080a3c15112b46819ac47dfe14e7c40a3f2caeff8226a WatchSource:0}: Error finding container 7431da64a25947f7b63080a3c15112b46819ac47dfe14e7c40a3f2caeff8226a: Status 404 returned error can't find the container with id 7431da64a25947f7b63080a3c15112b46819ac47dfe14e7c40a3f2caeff8226a
Jul 08 06:55:54.922959 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.922923    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dfcf58b95-bn599"]
Jul 08 06:55:54.925310 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.925287    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:54.928993 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.928973    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 06:55:54.930585 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.929263    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 06:55:54.930585 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.930164    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 06:55:54.930585 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.930440    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 06:55:54.930585 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.930507    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 06:55:54.930888 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.930622    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 06:55:54.930888 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.930760    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rg6rq\""
Jul 08 06:55:54.930888 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.930850    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 06:55:54.943602 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:54.943568    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfcf58b95-bn599"]
Jul 08 06:55:55.072347 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.072308    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-oauth-config\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.072515 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.072362    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-config\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.072515 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.072393    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-service-ca\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.072515 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.072424    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsm2j\" (UniqueName: \"kubernetes.io/projected/1e6a281d-ff8d-450c-b252-e9b0c319a152-kube-api-access-zsm2j\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.072515 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.072472    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-serving-cert\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.072775 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.072549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-oauth-serving-cert\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.097203 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.097169    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"]
Jul 08 06:55:55.099652 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.099628    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:55.103360 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.103336    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 08 06:55:55.103606 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.103591    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nxb9f\""
Jul 08 06:55:55.116950 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.116927    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"]
Jul 08 06:55:55.173125 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173053    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-serving-cert\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.173282 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173137    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-oauth-serving-cert\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.173282 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-oauth-config\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.173282 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/662f75e8-d26c-4975-8e9e-ba12328a648b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n6hv5\" (UID: \"662f75e8-d26c-4975-8e9e-ba12328a648b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:55.173282 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-config\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.173473 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-service-ca\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.173473 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.173335    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsm2j\" (UniqueName: \"kubernetes.io/projected/1e6a281d-ff8d-450c-b252-e9b0c319a152-kube-api-access-zsm2j\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.174844 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.174816    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-oauth-serving-cert\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.175419 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.175396    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-config\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.175970 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.175948    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-service-ca\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.176169 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.176150    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-serving-cert\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.177674 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.177632    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-oauth-config\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.191412 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.191389    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsm2j\" (UniqueName: \"kubernetes.io/projected/1e6a281d-ff8d-450c-b252-e9b0c319a152-kube-api-access-zsm2j\") pod \"console-dfcf58b95-bn599\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") " pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.238615 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.238576    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:55:55.273801 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.273762    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/662f75e8-d26c-4975-8e9e-ba12328a648b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n6hv5\" (UID: \"662f75e8-d26c-4975-8e9e-ba12328a648b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:55.277477 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.277388    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/662f75e8-d26c-4975-8e9e-ba12328a648b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n6hv5\" (UID: \"662f75e8-d26c-4975-8e9e-ba12328a648b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:55.304877 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.304817    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d" event={"ID":"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8","Type":"ContainerStarted","Data":"7431da64a25947f7b63080a3c15112b46819ac47dfe14e7c40a3f2caeff8226a"}
Jul 08 06:55:55.311831 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.311800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6" event={"ID":"f5bec0a8-4daa-4dc8-ad04-9b399c5f5826","Type":"ContainerStarted","Data":"899fd9b1e631af55fe0f2c76ae53f458b3191d6dfd8d6c774924267e0028211b"}
Jul 08 06:55:55.401351 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.401297    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qvvg6" podStartSLOduration=33.953116666 podStartE2EDuration="35.401278225s" totalImagesPullingTime="1.448161559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:55.333668787 +0000 UTC m=+82.932958489" watchObservedRunningTime="2026-07-08 06:55:55.401278225 +0000 UTC m=+83.000567927"
Jul 08 06:55:55.402181 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.402145    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfcf58b95-bn599"]
Jul 08 06:55:55.415907 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:55.415704    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:55.681778 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:55.681721    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6a281d_ff8d_450c_b252_e9b0c319a152.slice/crio-e63c63de86b8181abff5eae2cb53855dc1c998e60e54ed746794a8d6a163dc10 WatchSource:0}: Error finding container e63c63de86b8181abff5eae2cb53855dc1c998e60e54ed746794a8d6a163dc10: Status 404 returned error can't find the container with id e63c63de86b8181abff5eae2cb53855dc1c998e60e54ed746794a8d6a163dc10
Jul 08 06:55:56.316170 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:56.316130    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfcf58b95-bn599" event={"ID":"1e6a281d-ff8d-450c-b252-e9b0c319a152","Type":"ContainerStarted","Data":"e63c63de86b8181abff5eae2cb53855dc1c998e60e54ed746794a8d6a163dc10"}
Jul 08 06:55:56.586738 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:56.586637    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:56.589425 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:56.589387    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a81c997-4343-4bda-a495-4753336ff8bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-glfl6\" (UID: \"9a81c997-4343-4bda-a495-4753336ff8bb\") " pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:56.806242 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:56.806008    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h6mll\""
Jul 08 06:55:56.813480 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:56.813448    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-glfl6"
Jul 08 06:55:56.850034 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:56.850015    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"]
Jul 08 06:55:56.891653 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:56.891624    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662f75e8_d26c_4975_8e9e_ba12328a648b.slice/crio-275f3b43a63f62180da84b8f2aeba792b18a34c60fc5ee4aa2299ab221a4c488 WatchSource:0}: Error finding container 275f3b43a63f62180da84b8f2aeba792b18a34c60fc5ee4aa2299ab221a4c488: Status 404 returned error can't find the container with id 275f3b43a63f62180da84b8f2aeba792b18a34c60fc5ee4aa2299ab221a4c488
Jul 08 06:55:57.071480 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.070948    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-glfl6"]
Jul 08 06:55:57.075575 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:55:57.075547    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a81c997_4343_4bda_a495_4753336ff8bb.slice/crio-ec3ccf794c7968c158fe62e5af3cb006e19bc1a3786b57317207f6fd840feff6 WatchSource:0}: Error finding container ec3ccf794c7968c158fe62e5af3cb006e19bc1a3786b57317207f6fd840feff6: Status 404 returned error can't find the container with id ec3ccf794c7968c158fe62e5af3cb006e19bc1a3786b57317207f6fd840feff6
Jul 08 06:55:57.322063 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.322021    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5" event={"ID":"662f75e8-d26c-4975-8e9e-ba12328a648b","Type":"ContainerStarted","Data":"275f3b43a63f62180da84b8f2aeba792b18a34c60fc5ee4aa2299ab221a4c488"}
Jul 08 06:55:57.323670 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.323633    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp" event={"ID":"43230123-d998-4f29-9dbe-104648246888","Type":"ContainerStarted","Data":"cae783ba6b5dd4847f97ececb3d92f972270901f87a0b800d3567ee63cefebd3"}
Jul 08 06:55:57.325618 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.325590    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d" event={"ID":"ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8","Type":"ContainerStarted","Data":"b8a8a2d4774909b6e0d5470450222879488c1200c5e38126774309114585d93b"}
Jul 08 06:55:57.326981 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.326954    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glfl6" event={"ID":"9a81c997-4343-4bda-a495-4753336ff8bb","Type":"ContainerStarted","Data":"ec3ccf794c7968c158fe62e5af3cb006e19bc1a3786b57317207f6fd840feff6"}
Jul 08 06:55:57.342573 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.342523    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-849686c7c8-lthjp" podStartSLOduration=1.023802695 podStartE2EDuration="4.342504173s" totalImagesPullingTime="3.318701478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:57.341256684 +0000 UTC m=+84.940546388" watchObservedRunningTime="2026-07-08 06:55:57.342504173 +0000 UTC m=+84.941793879"
Jul 08 06:55:57.361264 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.361126    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qxs7d" podStartSLOduration=34.177212882 podStartE2EDuration="36.361107014s" totalImagesPullingTime="2.183894132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:57.360541952 +0000 UTC m=+84.959831679" watchObservedRunningTime="2026-07-08 06:55:57.361107014 +0000 UTC m=+84.960396718"
Jul 08 06:55:57.827734 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.827678    2571 patch_prober.go:28] interesting pod/image-registry-7bd9657ff9-lv66b container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 06:55:57.827903 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:57.827759    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" podUID="4077e488-ca01-4396-9f45-cebd5fbdcef1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 06:55:59.244460 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:59.244423    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:55:59.337062 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:59.337029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5" event={"ID":"662f75e8-d26c-4975-8e9e-ba12328a648b","Type":"ContainerStarted","Data":"7031d5e0ef1793e7ef7458e8f36bcffd37091ede8e65cc81b010cd3c91a984d9"}
Jul 08 06:55:59.337258 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:59.337237    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:59.346363 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:59.346338    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5"
Jul 08 06:55:59.368643 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:55:59.368597    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n6hv5" podStartSLOduration=2.9140590140000002 podStartE2EDuration="4.368582532s" totalImagesPullingTime="1.454523518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:55:59.36689603 +0000 UTC m=+86.966185757" watchObservedRunningTime="2026-07-08 06:55:59.368582532 +0000 UTC m=+86.967872213"
Jul 08 06:56:00.158066 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.158036    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"]
Jul 08 06:56:00.170176 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.170143    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.172388 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.172359    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"]
Jul 08 06:56:00.173143 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.173005    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 08 06:56:00.173143 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.173008    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 08 06:56:00.173143 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.173038    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 06:56:00.174027 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.174006    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xckzt\""
Jul 08 06:56:00.323704 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.323649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a65b84cb-1739-4de2-aa09-a9b7e61e6705-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.324152 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.323728    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a65b84cb-1739-4de2-aa09-a9b7e61e6705-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.324152 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.323755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a65b84cb-1739-4de2-aa09-a9b7e61e6705-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.324152 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.323815    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q7lw\" (UniqueName: \"kubernetes.io/projected/a65b84cb-1739-4de2-aa09-a9b7e61e6705-kube-api-access-2q7lw\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.424305 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.424227    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q7lw\" (UniqueName: \"kubernetes.io/projected/a65b84cb-1739-4de2-aa09-a9b7e61e6705-kube-api-access-2q7lw\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.424305 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.424292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a65b84cb-1739-4de2-aa09-a9b7e61e6705-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.424535 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.424338    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a65b84cb-1739-4de2-aa09-a9b7e61e6705-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.424535 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.424365    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a65b84cb-1739-4de2-aa09-a9b7e61e6705-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.425198 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.425110    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a65b84cb-1739-4de2-aa09-a9b7e61e6705-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.427294 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.427271    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a65b84cb-1739-4de2-aa09-a9b7e61e6705-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.427432 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.427412    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a65b84cb-1739-4de2-aa09-a9b7e61e6705-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.433988 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.433963    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q7lw\" (UniqueName: \"kubernetes.io/projected/a65b84cb-1739-4de2-aa09-a9b7e61e6705-kube-api-access-2q7lw\") pod \"prometheus-operator-567f7b8b7c-7w546\" (UID: \"a65b84cb-1739-4de2-aa09-a9b7e61e6705\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:00.483806 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:00.483767    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"
Jul 08 06:56:01.954858 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:01.954820    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7w546"]
Jul 08 06:56:02.348880 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:02.348836    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546" event={"ID":"a65b84cb-1739-4de2-aa09-a9b7e61e6705","Type":"ContainerStarted","Data":"1c7b83665581d5385a772e08e092f16829b92e8c762d0ef2ce5cea4d67138cce"}
Jul 08 06:56:02.350347 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:02.350316    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfcf58b95-bn599" event={"ID":"1e6a281d-ff8d-450c-b252-e9b0c319a152","Type":"ContainerStarted","Data":"860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588"}
Jul 08 06:56:02.351819 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:02.351800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glfl6" event={"ID":"9a81c997-4343-4bda-a495-4753336ff8bb","Type":"ContainerStarted","Data":"fe04d768bfa81774cd8d0ec703cd421b72d678aea75e2388f151181dd6df5756"}
Jul 08 06:56:02.372321 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:02.372267    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dfcf58b95-bn599" podStartSLOduration=2.258608959 podStartE2EDuration="8.372251098s" totalImagesPullingTime="6.113642139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:02.371044039 +0000 UTC m=+89.970333776" watchObservedRunningTime="2026-07-08 06:56:02.372251098 +0000 UTC m=+89.971540801"
Jul 08 06:56:03.358349 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:03.358309    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glfl6" event={"ID":"9a81c997-4343-4bda-a495-4753336ff8bb","Type":"ContainerStarted","Data":"098494b2b519e5ce3b92ddd20608980f835925b98ae5e47d618987b4a394d96a"}
Jul 08 06:56:04.365485 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:04.365450    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546" event={"ID":"a65b84cb-1739-4de2-aa09-a9b7e61e6705","Type":"ContainerStarted","Data":"e6d1856e90c2f6937b4855096b7498437885daf4528bd2261d8edb588c8f38ba"}
Jul 08 06:56:04.365969 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:04.365515    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546" event={"ID":"a65b84cb-1739-4de2-aa09-a9b7e61e6705","Type":"ContainerStarted","Data":"c7ca74ac09fa36cf2a8bdf2fad720d23bdc35388ee1363d00606e4f71dcf35ab"}
Jul 08 06:56:04.406702 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:04.406515    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7w546" podStartSLOduration=2.963968527 podStartE2EDuration="4.406495535s" totalImagesPullingTime="1.442527008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:04.405761649 +0000 UTC m=+92.005051354" watchObservedRunningTime="2026-07-08 06:56:04.406495535 +0000 UTC m=+92.005785240"
Jul 08 06:56:05.239067 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:05.239022    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:56:05.239186 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:05.239087    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:56:05.243710 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:05.243661    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:56:05.371112 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:05.371034    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-glfl6" event={"ID":"9a81c997-4343-4bda-a495-4753336ff8bb","Type":"ContainerStarted","Data":"5789a4b5c012f37bcc64e3343696264717684b4d4c0c08d1eabb0e4d02ebcc3d"}
Jul 08 06:56:05.374969 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:05.374949    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:56:05.391496 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:05.391453    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-glfl6" podStartSLOduration=33.502097077 podStartE2EDuration="41.391439207s" totalImagesPullingTime="7.88934213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 06:55:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:05.390775487 +0000 UTC m=+92.990065189" watchObservedRunningTime="2026-07-08 06:56:05.391439207 +0000 UTC m=+92.990728928"
Jul 08 06:56:06.753908 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.753879    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7tjbk"]
Jul 08 06:56:06.757346 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.757324    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.760214 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.760194    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 06:56:06.760335 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.760194    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 06:56:06.760388 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.760349    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zvxdg\""
Jul 08 06:56:06.764201 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.764185    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 06:56:06.883453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883425    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-sys\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883629 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883472    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-textfile\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883629 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883495    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-tls\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883629 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883575    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-wtmp\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883629 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77423e1e-a673-4ff5-aa50-d220f54dab03-metrics-client-ca\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883826 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883717    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-root\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883826 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883739    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883826 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883759    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-accelerators-collector-config\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.883826 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.883793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dfvh\" (UniqueName: \"kubernetes.io/projected/77423e1e-a673-4ff5-aa50-d220f54dab03-kube-api-access-6dfvh\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985190 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985161    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-textfile\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985195    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-tls\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985228    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-wtmp\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985252    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77423e1e-a673-4ff5-aa50-d220f54dab03-metrics-client-ca\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985357 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985348    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-root\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985380    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985385    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-wtmp\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-accelerators-collector-config\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985456    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dfvh\" (UniqueName: \"kubernetes.io/projected/77423e1e-a673-4ff5-aa50-d220f54dab03-kube-api-access-6dfvh\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985461    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-root\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-sys\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985521    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77423e1e-a673-4ff5-aa50-d220f54dab03-sys\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985569 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985557    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-textfile\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.985976 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.985928    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77423e1e-a673-4ff5-aa50-d220f54dab03-metrics-client-ca\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.986034 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.986013    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-accelerators-collector-config\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.987782 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.987762    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:06.987879 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:06.987861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77423e1e-a673-4ff5-aa50-d220f54dab03-node-exporter-tls\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:07.013553 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:07.013490    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dfvh\" (UniqueName: \"kubernetes.io/projected/77423e1e-a673-4ff5-aa50-d220f54dab03-kube-api-access-6dfvh\") pod \"node-exporter-7tjbk\" (UID: \"77423e1e-a673-4ff5-aa50-d220f54dab03\") " pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:07.066185 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:07.066155    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7tjbk"
Jul 08 06:56:07.077975 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:56:07.077949    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77423e1e_a673_4ff5_aa50_d220f54dab03.slice/crio-b3e138ecf38b9682f45070339c0e301a47e9fb3b3d2f4cd5a3a0ba967eb3cff0 WatchSource:0}: Error finding container b3e138ecf38b9682f45070339c0e301a47e9fb3b3d2f4cd5a3a0ba967eb3cff0: Status 404 returned error can't find the container with id b3e138ecf38b9682f45070339c0e301a47e9fb3b3d2f4cd5a3a0ba967eb3cff0
Jul 08 06:56:07.379073 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:07.379040    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7tjbk" event={"ID":"77423e1e-a673-4ff5-aa50-d220f54dab03","Type":"ContainerStarted","Data":"b3e138ecf38b9682f45070339c0e301a47e9fb3b3d2f4cd5a3a0ba967eb3cff0"}
Jul 08 06:56:08.263669 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.263642    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:56:08.267284 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.267253    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.274369 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.274038    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 08 06:56:08.275074 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.275048    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 08 06:56:08.275291 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.275275    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 08 06:56:08.276184 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.275762    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 08 06:56:08.276184 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.276043    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 08 06:56:08.277920 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.277022    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 08 06:56:08.277920 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.277758    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7tg6l\""
Jul 08 06:56:08.286662 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.286632    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 08 06:56:08.289237 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.289215    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 08 06:56:08.301488 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.301464    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 08 06:56:08.384201 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.384165    2571 generic.go:356] "Generic (PLEG): container finished" podID="77423e1e-a673-4ff5-aa50-d220f54dab03" containerID="e8edffac873ef351b691e07fd0cfe2dec97cd44c0fa76fa29fca4a0a9aeb871e" exitCode=0
Jul 08 06:56:08.384339 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.384218    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7tjbk" event={"ID":"77423e1e-a673-4ff5-aa50-d220f54dab03","Type":"ContainerDied","Data":"e8edffac873ef351b691e07fd0cfe2dec97cd44c0fa76fa29fca4a0a9aeb871e"}
Jul 08 06:56:08.398397 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398377    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-out\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398507 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398407    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398507 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398430    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398507 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jv5g\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-kube-api-access-7jv5g\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398580    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398612    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398834 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398834 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398773    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-volume\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.398834 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398795    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.399009 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398837    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-web-config\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.399009 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398873    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.399009 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.398934    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499397 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499361    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499537 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-volume\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499537 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499518    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-web-config\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499667 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499606    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499832 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499677    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-out\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499832 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499832 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499832 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499778    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jv5g\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-kube-api-access-7jv5g\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.499832 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499827    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.500095 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499868    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.500095 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.499894    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.500200 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.500090    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.500904 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.500879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.502223 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.502165    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.503203 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.502664    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-web-config\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.503203 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.503141    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-volume\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.503203 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.503168    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.503346 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.503217    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.503428 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.503408    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-out\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.504013 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.503993    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.504929 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.504900    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.505027 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.505006    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.505360 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.505344    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.511744 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.511725    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfcf58b95-bn599"]
Jul 08 06:56:08.602634 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.602599    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jv5g\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-kube-api-access-7jv5g\") pod \"alertmanager-main-0\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:08.668066 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.668028    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:56:08.878297 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:08.878263    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:56:09.042843 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:56:09.042812    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d03695_168e_40bf_a0d4_1be35ca18cd6.slice/crio-8d36d715bf5cb0b602c5e53971544ae31fde0ed5e12cb55f8b60e9ee5b1b6d93 WatchSource:0}: Error finding container 8d36d715bf5cb0b602c5e53971544ae31fde0ed5e12cb55f8b60e9ee5b1b6d93: Status 404 returned error can't find the container with id 8d36d715bf5cb0b602c5e53971544ae31fde0ed5e12cb55f8b60e9ee5b1b6d93
Jul 08 06:56:09.045639 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:09.045615    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:56:09.390501 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:09.390461    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7tjbk" event={"ID":"77423e1e-a673-4ff5-aa50-d220f54dab03","Type":"ContainerStarted","Data":"a037c621e844b8eb1eddbf92793f92835991e4d11b61c4bdad1995f0630d6016"}
Jul 08 06:56:09.391014 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:09.390532    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7tjbk" event={"ID":"77423e1e-a673-4ff5-aa50-d220f54dab03","Type":"ContainerStarted","Data":"4903f11b15fe36a3f1ae7f80f1fe4293c0e692aed92ddcc7635995c315264c59"}
Jul 08 06:56:09.391739 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:09.391713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"8d36d715bf5cb0b602c5e53971544ae31fde0ed5e12cb55f8b60e9ee5b1b6d93"}
Jul 08 06:56:09.426456 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:09.426399    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7tjbk" podStartSLOduration=2.260252489 podStartE2EDuration="3.426378428s" totalImagesPullingTime="1.089146204s" totalInitContainerRuntime="76.979735ms" isStatefulPod=true podCreationTimestamp="2026-07-08 06:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:09.425250035 +0000 UTC m=+97.024539739" watchObservedRunningTime="2026-07-08 06:56:09.426378428 +0000 UTC m=+97.025668132"
Jul 08 06:56:10.397770 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:10.397735    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74" exitCode=0
Jul 08 06:56:10.398131 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:10.397822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"}
Jul 08 06:56:11.381516 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.381474    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-744f84b76c-rj9cq"]
Jul 08 06:56:11.384911 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.384884    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.388157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.388122    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 06:56:11.388827 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.388805    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 08 06:56:11.389061 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.388823    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zr64n\""
Jul 08 06:56:11.389139 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.389075    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 08 06:56:11.389225 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.389207    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 08 06:56:11.399705 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.399656    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4ngn1hedsccb5\""
Jul 08 06:56:11.425029 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.424996    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-secret-metrics-server-tls\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.425029 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.425025    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c037f44f-2206-465a-bfe4-5a0864515456-metrics-server-audit-profiles\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.425225 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.425043    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wk89\" (UniqueName: \"kubernetes.io/projected/c037f44f-2206-465a-bfe4-5a0864515456-kube-api-access-7wk89\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.425225 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.425068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-secret-metrics-server-client-certs\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.425225 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.425138    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c037f44f-2206-465a-bfe4-5a0864515456-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.425225 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.425188    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c037f44f-2206-465a-bfe4-5a0864515456-audit-log\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.425225 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.425206    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-client-ca-bundle\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.470772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.470675    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-744f84b76c-rj9cq"]
Jul 08 06:56:11.526558 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526514    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-secret-metrics-server-client-certs\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.526768 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526581    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c037f44f-2206-465a-bfe4-5a0864515456-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.526845 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526789    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c037f44f-2206-465a-bfe4-5a0864515456-audit-log\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.526845 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-client-ca-bundle\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.526951 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-secret-metrics-server-tls\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.526951 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c037f44f-2206-465a-bfe4-5a0864515456-metrics-server-audit-profiles\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.526951 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.526916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wk89\" (UniqueName: \"kubernetes.io/projected/c037f44f-2206-465a-bfe4-5a0864515456-kube-api-access-7wk89\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.527585 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.527553    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c037f44f-2206-465a-bfe4-5a0864515456-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.528088 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.528057    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c037f44f-2206-465a-bfe4-5a0864515456-audit-log\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.528190 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.528127    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c037f44f-2206-465a-bfe4-5a0864515456-metrics-server-audit-profiles\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.529577 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.529553    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-secret-metrics-server-client-certs\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.529808 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.529790    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-client-ca-bundle\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.529976 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.529921    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c037f44f-2206-465a-bfe4-5a0864515456-secret-metrics-server-tls\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.539147 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.539125    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wk89\" (UniqueName: \"kubernetes.io/projected/c037f44f-2206-465a-bfe4-5a0864515456-kube-api-access-7wk89\") pod \"metrics-server-744f84b76c-rj9cq\" (UID: \"c037f44f-2206-465a-bfe4-5a0864515456\") " pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.697301 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.697274    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:11.840406 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:11.840371    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-744f84b76c-rj9cq"]
Jul 08 06:56:12.108580 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:56:12.108551    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc037f44f_2206_465a_bfe4_5a0864515456.slice/crio-289bf9d22e82a6fe22ef7ba52b1d4cbcb9628a6b8dce1ad1c08ed7d32d2da465 WatchSource:0}: Error finding container 289bf9d22e82a6fe22ef7ba52b1d4cbcb9628a6b8dce1ad1c08ed7d32d2da465: Status 404 returned error can't find the container with id 289bf9d22e82a6fe22ef7ba52b1d4cbcb9628a6b8dce1ad1c08ed7d32d2da465
Jul 08 06:56:12.409602 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:12.409578    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"}
Jul 08 06:56:12.409920 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:12.409666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"}
Jul 08 06:56:12.409920 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:12.409735    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"}
Jul 08 06:56:12.410726 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:12.410672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq" event={"ID":"c037f44f-2206-465a-bfe4-5a0864515456","Type":"ContainerStarted","Data":"289bf9d22e82a6fe22ef7ba52b1d4cbcb9628a6b8dce1ad1c08ed7d32d2da465"}
Jul 08 06:56:13.321396 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.321367    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:56:13.348639 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.348606    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:56:13.348833 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.348769    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.354157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.353472    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 06:56:13.354157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.353803    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 06:56:13.354157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.353985    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 06:56:13.354157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.354058    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 06:56:13.354157 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.354182    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8scvq\""
Jul 08 06:56:13.354608 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.354312    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 06:56:13.354608 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.354468    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 06:56:13.355404 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.355026    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 06:56:13.355404 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.355042    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 06:56:13.360877 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.358781    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-n77s5p8qvh3b\""
Jul 08 06:56:13.360877 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.360295    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 06:56:13.361909 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.361889    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 06:56:13.362406 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.362386    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 06:56:13.362593 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.362578    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 06:56:13.368978 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.368952    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 06:56:13.369511 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.369480    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 06:56:13.420280 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.420241    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"}
Jul 08 06:56:13.420660 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.420353    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"}
Jul 08 06:56:13.447529 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.447751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447541    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.447751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447578    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.447751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447642    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.447751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447667    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.447751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdf5w\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-kube-api-access-gdf5w\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447809    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-config\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447833    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447920    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.447969    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448367 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.448032    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448367 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.448132    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448367 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.448170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448557 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.448540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448602 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.448587    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.448719 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.448670    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.549906 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.549849    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.549906 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.549903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550129 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.549936    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550129 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.549958    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550129 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550021    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550254 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550254 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550205    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550254 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550246    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550308    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550385    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdf5w\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-kube-api-access-gdf5w\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550410    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-config\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550440    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550480    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550528    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550551    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550979 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550597    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550979 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550755    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.550979 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.550755    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.551275 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.551249    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.551852 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.551826    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.552420 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.552331    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.553234 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.553210    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.553785 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.553764    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.554479 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.553929    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.554479 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.554199    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.555395 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.555369    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.556202 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.556178    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.556288 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.556245    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.560971 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.560925    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.561073 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.561016    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.561073 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.561053    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.561306 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.561284    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.561461 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.561430    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.561726 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.561709    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-config\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.562538 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.562505    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.565789 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.565765    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdf5w\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-kube-api-access-gdf5w\") pod \"prometheus-k8s-0\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:13.669986 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:13.669951    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:14.262903 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:14.262870    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lb9c2"
Jul 08 06:56:14.554338 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:14.554310    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:56:14.557604 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:56:14.557580    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f248415_02b3_452d_8e0f_b0ca32051a27.slice/crio-fe06b5dd8f5ae4c6bbd5e14089c550fc5f2f7925c2c0f5dfea163b8268723c32 WatchSource:0}: Error finding container fe06b5dd8f5ae4c6bbd5e14089c550fc5f2f7925c2c0f5dfea163b8268723c32: Status 404 returned error can't find the container with id fe06b5dd8f5ae4c6bbd5e14089c550fc5f2f7925c2c0f5dfea163b8268723c32
Jul 08 06:56:15.275946 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.275903    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bd9657ff9-lv66b"]
Jul 08 06:56:15.429657 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.429628    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794" exitCode=0
Jul 08 06:56:15.429867 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.429717    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"}
Jul 08 06:56:15.429867 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.429784    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"fe06b5dd8f5ae4c6bbd5e14089c550fc5f2f7925c2c0f5dfea163b8268723c32"}
Jul 08 06:56:15.431254 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.431230    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq" event={"ID":"c037f44f-2206-465a-bfe4-5a0864515456","Type":"ContainerStarted","Data":"52e9bd554d2ddd4cc7b0b60630b2523ad6c9f5e0f6f691d9ed9807af4fc3c6e6"}
Jul 08 06:56:15.434803 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.434779    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerStarted","Data":"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"}
Jul 08 06:56:15.481587 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.481538    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq" podStartSLOduration=2.180067673 podStartE2EDuration="4.481520599s" totalImagesPullingTime="2.301452926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:15.481350762 +0000 UTC m=+103.080640479" watchObservedRunningTime="2026-07-08 06:56:15.481520599 +0000 UTC m=+103.080810307"
Jul 08 06:56:15.508932 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:15.508881    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.630961902 podStartE2EDuration="7.508866927s" totalImagesPullingTime="4.860221463s" totalInitContainerRuntime="17.683562ms" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:15.50702594 +0000 UTC m=+103.106315643" watchObservedRunningTime="2026-07-08 06:56:15.508866927 +0000 UTC m=+103.108156631"
Jul 08 06:56:19.456634 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:19.456597    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"}
Jul 08 06:56:19.457063 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:19.456673    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"}
Jul 08 06:56:21.468270 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:21.468229    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"}
Jul 08 06:56:21.468771 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:21.468297    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"}
Jul 08 06:56:21.468771 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:21.468332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"}
Jul 08 06:56:21.468771 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:21.468357    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerStarted","Data":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"}
Jul 08 06:56:21.499641 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:21.499594    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.264331426 podStartE2EDuration="8.49957963s" totalImagesPullingTime="5.214957481s" totalInitContainerRuntime="20.290723ms" isStatefulPod=false podCreationTimestamp="2026-07-08 06:56:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:21.497750336 +0000 UTC m=+109.097040040" watchObservedRunningTime="2026-07-08 06:56:21.49957963 +0000 UTC m=+109.098869332"
Jul 08 06:56:23.670747 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:23.670677    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:56:29.507704 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:29.507589    2571 generic.go:356] "Generic (PLEG): container finished" podID="00afd1f2-a8a8-4192-861f-733bc074b996" containerID="3863493dbc3a528cf4c41f01e1a826cbbb857c129a1f93f89862009683e6b21a" exitCode=0
Jul 08 06:56:29.507704 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:29.507666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf" event={"ID":"00afd1f2-a8a8-4192-861f-733bc074b996","Type":"ContainerDied","Data":"3863493dbc3a528cf4c41f01e1a826cbbb857c129a1f93f89862009683e6b21a"}
Jul 08 06:56:29.508153 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:29.508087    2571 scope.go:117] "RemoveContainer" containerID="3863493dbc3a528cf4c41f01e1a826cbbb857c129a1f93f89862009683e6b21a"
Jul 08 06:56:29.509469 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:29.509446    2571 generic.go:356] "Generic (PLEG): container finished" podID="09c5f560-1820-422f-823b-7c90342c0793" containerID="eb95962fc983aa3462ef304c513233aabd83f3e6374416996b43a49ece09b0bd" exitCode=0
Jul 08 06:56:29.509549 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:29.509488    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m" event={"ID":"09c5f560-1820-422f-823b-7c90342c0793","Type":"ContainerDied","Data":"eb95962fc983aa3462ef304c513233aabd83f3e6374416996b43a49ece09b0bd"}
Jul 08 06:56:29.509756 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:29.509744    2571 scope.go:117] "RemoveContainer" containerID="eb95962fc983aa3462ef304c513233aabd83f3e6374416996b43a49ece09b0bd"
Jul 08 06:56:30.515802 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:30.515760    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-g2pqf" event={"ID":"00afd1f2-a8a8-4192-861f-733bc074b996","Type":"ContainerStarted","Data":"a600e9632bf732391edab9036dbe084a69ce650ca7318035549201a0d6ebd807"}
Jul 08 06:56:30.517751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:30.517718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hwn2m" event={"ID":"09c5f560-1820-422f-823b-7c90342c0793","Type":"ContainerStarted","Data":"c0191be394845ebd5ca8aa1791cf773c7537fb42f968603c9078e7edbac3cd21"}
Jul 08 06:56:31.697483 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:31.697446    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:31.697902 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:31.697537    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:33.532236 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:33.532176    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-dfcf58b95-bn599" podUID="1e6a281d-ff8d-450c-b252-e9b0c319a152" containerName="console" containerID="cri-o://860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588" gracePeriod=15
Jul 08 06:56:34.307327 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.307305    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:56:34.458517 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.458441    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-service-ca\") pod \"1e6a281d-ff8d-450c-b252-e9b0c319a152\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") "
Jul 08 06:56:34.458517 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.458497    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-config\") pod \"1e6a281d-ff8d-450c-b252-e9b0c319a152\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") "
Jul 08 06:56:34.458752 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.458533    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsm2j\" (UniqueName: \"kubernetes.io/projected/1e6a281d-ff8d-450c-b252-e9b0c319a152-kube-api-access-zsm2j\") pod \"1e6a281d-ff8d-450c-b252-e9b0c319a152\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") "
Jul 08 06:56:34.458752 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.458570    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-serving-cert\") pod \"1e6a281d-ff8d-450c-b252-e9b0c319a152\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") "
Jul 08 06:56:34.458752 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.458611    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-oauth-serving-cert\") pod \"1e6a281d-ff8d-450c-b252-e9b0c319a152\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") "
Jul 08 06:56:34.458752 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.458679    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-oauth-config\") pod \"1e6a281d-ff8d-450c-b252-e9b0c319a152\" (UID: \"1e6a281d-ff8d-450c-b252-e9b0c319a152\") "
Jul 08 06:56:34.459024 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.459000    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-service-ca" (OuterVolumeSpecName: "service-ca") pod "1e6a281d-ff8d-450c-b252-e9b0c319a152" (UID: "1e6a281d-ff8d-450c-b252-e9b0c319a152"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:34.459092 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.459035    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-config" (OuterVolumeSpecName: "console-config") pod "1e6a281d-ff8d-450c-b252-e9b0c319a152" (UID: "1e6a281d-ff8d-450c-b252-e9b0c319a152"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:34.459144 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.459081    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1e6a281d-ff8d-450c-b252-e9b0c319a152" (UID: "1e6a281d-ff8d-450c-b252-e9b0c319a152"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:34.461129 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.461104    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1e6a281d-ff8d-450c-b252-e9b0c319a152" (UID: "1e6a281d-ff8d-450c-b252-e9b0c319a152"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:56:34.461236 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.461136    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1e6a281d-ff8d-450c-b252-e9b0c319a152" (UID: "1e6a281d-ff8d-450c-b252-e9b0c319a152"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:56:34.461236 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.461158    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6a281d-ff8d-450c-b252-e9b0c319a152-kube-api-access-zsm2j" (OuterVolumeSpecName: "kube-api-access-zsm2j") pod "1e6a281d-ff8d-450c-b252-e9b0c319a152" (UID: "1e6a281d-ff8d-450c-b252-e9b0c319a152"). InnerVolumeSpecName "kube-api-access-zsm2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:34.533998 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.533966    2571 generic.go:356] "Generic (PLEG): container finished" podID="1e6a281d-ff8d-450c-b252-e9b0c319a152" containerID="860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588" exitCode=0
Jul 08 06:56:34.534323 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.534013    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfcf58b95-bn599"
Jul 08 06:56:34.534323 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.534051    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfcf58b95-bn599" event={"ID":"1e6a281d-ff8d-450c-b252-e9b0c319a152","Type":"ContainerDied","Data":"860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588"}
Jul 08 06:56:34.534323 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.534123    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfcf58b95-bn599" event={"ID":"1e6a281d-ff8d-450c-b252-e9b0c319a152","Type":"ContainerDied","Data":"e63c63de86b8181abff5eae2cb53855dc1c998e60e54ed746794a8d6a163dc10"}
Jul 08 06:56:34.534323 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.534142    2571 scope.go:117] "RemoveContainer" containerID="860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588"
Jul 08 06:56:34.550898 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.550881    2571 scope.go:117] "RemoveContainer" containerID="860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588"
Jul 08 06:56:34.551178 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:56:34.551160    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588\": container with ID starting with 860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588 not found: ID does not exist" containerID="860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588"
Jul 08 06:56:34.551239 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.551187    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588"} err="failed to get container status \"860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588\": rpc error: code = NotFound desc = could not find container \"860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588\": container with ID starting with 860323bdcc0d42f5093f21da11a5c190aad5c261e35c325658ff1ee3e116b588 not found: ID does not exist"
Jul 08 06:56:34.555394 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.555374    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfcf58b95-bn599"]
Jul 08 06:56:34.559342 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.559322    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-service-ca\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:34.559342 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.559344    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-config\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:34.559459 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.559353    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsm2j\" (UniqueName: \"kubernetes.io/projected/1e6a281d-ff8d-450c-b252-e9b0c319a152-kube-api-access-zsm2j\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:34.559459 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.559363    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-serving-cert\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:34.559459 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.559373    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e6a281d-ff8d-450c-b252-e9b0c319a152-oauth-serving-cert\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:34.559459 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.559381    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e6a281d-ff8d-450c-b252-e9b0c319a152-console-oauth-config\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:34.560019 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.560002    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dfcf58b95-bn599"]
Jul 08 06:56:34.913366 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:34.913331    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e6a281d-ff8d-450c-b252-e9b0c319a152" path="/var/lib/kubelet/pods/1e6a281d-ff8d-450c-b252-e9b0c319a152/volumes"
Jul 08 06:56:40.296650 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.296590    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" podUID="4077e488-ca01-4396-9f45-cebd5fbdcef1" containerName="registry" containerID="cri-o://853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb" gracePeriod=30
Jul 08 06:56:40.556143 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.556089    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:56:40.558826 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.558800    2571 generic.go:356] "Generic (PLEG): container finished" podID="4077e488-ca01-4396-9f45-cebd5fbdcef1" containerID="853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb" exitCode=0
Jul 08 06:56:40.558925 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.558836    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b"
Jul 08 06:56:40.558925 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.558873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" event={"ID":"4077e488-ca01-4396-9f45-cebd5fbdcef1","Type":"ContainerDied","Data":"853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb"}
Jul 08 06:56:40.558925 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.558913    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bd9657ff9-lv66b" event={"ID":"4077e488-ca01-4396-9f45-cebd5fbdcef1","Type":"ContainerDied","Data":"2c8bc61e6cfcb9af9326c47ce4502a3ed6b54de1cf1084e6f3380e4ce2015fa5"}
Jul 08 06:56:40.559019 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.558938    2571 scope.go:117] "RemoveContainer" containerID="853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb"
Jul 08 06:56:40.567041 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.567027    2571 scope.go:117] "RemoveContainer" containerID="853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb"
Jul 08 06:56:40.567300 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:56:40.567281    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb\": container with ID starting with 853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb not found: ID does not exist" containerID="853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb"
Jul 08 06:56:40.567344 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.567307    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb"} err="failed to get container status \"853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb\": rpc error: code = NotFound desc = could not find container \"853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb\": container with ID starting with 853f24fbd920533e4145d4e053fa29eee7e7276cfae05123d8758c0be6fc8fdb not found: ID does not exist"
Jul 08 06:56:40.715215 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715181    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-certificates\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715241    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-trusted-ca\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715272    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4077e488-ca01-4396-9f45-cebd5fbdcef1-ca-trust-extracted\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715327    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-image-registry-private-configuration\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715391 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715365    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715595 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715411    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-installation-pull-secrets\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715595 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715437    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-bound-sa-token\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715595 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715468    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd8rk\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-kube-api-access-pd8rk\") pod \"4077e488-ca01-4396-9f45-cebd5fbdcef1\" (UID: \"4077e488-ca01-4396-9f45-cebd5fbdcef1\") "
Jul 08 06:56:40.715767 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715653    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:40.715827 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715773    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-certificates\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.715879 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.715818    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:56:40.718607 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.718554    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:56:40.718607 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.718555    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:40.718607 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.718600    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:40.718864 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.718635    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:56:40.718864 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.718785    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-kube-api-access-pd8rk" (OuterVolumeSpecName: "kube-api-access-pd8rk") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "kube-api-access-pd8rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:56:40.724895 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.724871    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4077e488-ca01-4396-9f45-cebd5fbdcef1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4077e488-ca01-4396-9f45-cebd5fbdcef1" (UID: "4077e488-ca01-4396-9f45-cebd5fbdcef1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816372    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-installation-pull-secrets\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816400    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-bound-sa-token\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816409    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd8rk\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-kube-api-access-pd8rk\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816419    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4077e488-ca01-4396-9f45-cebd5fbdcef1-trusted-ca\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816429    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4077e488-ca01-4396-9f45-cebd5fbdcef1-ca-trust-extracted\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816437    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4077e488-ca01-4396-9f45-cebd5fbdcef1-image-registry-private-configuration\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.816453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.816448    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4077e488-ca01-4396-9f45-cebd5fbdcef1-registry-tls\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:56:40.881935 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.881906    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bd9657ff9-lv66b"]
Jul 08 06:56:40.885453 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.885430    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7bd9657ff9-lv66b"]
Jul 08 06:56:40.912831 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:40.912804    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4077e488-ca01-4396-9f45-cebd5fbdcef1" path="/var/lib/kubelet/pods/4077e488-ca01-4396-9f45-cebd5fbdcef1/volumes"
Jul 08 06:56:41.723343 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:41.723292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:56:41.725881 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:41.725859    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624-metrics-certs\") pod \"network-metrics-daemon-vtskv\" (UID: \"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624\") " pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:56:41.941351 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:41.941317    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4f688\""
Jul 08 06:56:41.949251 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:41.949229    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vtskv"
Jul 08 06:56:42.096927 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:42.096904    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vtskv"]
Jul 08 06:56:42.099354 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:56:42.099323    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2cb3c0_f5fd_4ac8_b07e_7e7d3dec8624.slice/crio-5ed3815c8e8ecdbca5677c2da2147541c6fcb3d7005d80eebea4e07c64aa3567 WatchSource:0}: Error finding container 5ed3815c8e8ecdbca5677c2da2147541c6fcb3d7005d80eebea4e07c64aa3567: Status 404 returned error can't find the container with id 5ed3815c8e8ecdbca5677c2da2147541c6fcb3d7005d80eebea4e07c64aa3567
Jul 08 06:56:42.570717 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:42.570641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vtskv" event={"ID":"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624","Type":"ContainerStarted","Data":"5ed3815c8e8ecdbca5677c2da2147541c6fcb3d7005d80eebea4e07c64aa3567"}
Jul 08 06:56:43.576658 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:43.576614    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vtskv" event={"ID":"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624","Type":"ContainerStarted","Data":"ae4532901a323535289bcd47d0ba55ef1e3f174594e004797f60bcdcda2c24f1"}
Jul 08 06:56:43.577052 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:43.576736    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vtskv" event={"ID":"2a2cb3c0-f5fd-4ac8-b07e-7e7d3dec8624","Type":"ContainerStarted","Data":"87e60aada584d420cb08ffe83751e6059802e34729b02ee5f65e125c46eb5e22"}
Jul 08 06:56:43.595675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:43.595627    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vtskv" podStartSLOduration=130.697778232 podStartE2EDuration="2m11.595612427s" totalImagesPullingTime="897.834195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:56:43.59388934 +0000 UTC m=+131.193179044" watchObservedRunningTime="2026-07-08 06:56:43.595612427 +0000 UTC m=+131.194902129"
Jul 08 06:56:51.703494 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:51.703465    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:56:51.707342 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:56:51.707318    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-744f84b76c-rj9cq"
Jul 08 06:57:13.670816 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:13.670782    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:13.690658 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:13.690630    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:13.725422 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:13.725382    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:27.299951 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.299865    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:57:27.751733 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.751645    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="alertmanager" containerID="cri-o://16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7" gracePeriod=120
Jul 08 06:57:27.751733 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.751708    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-metric" containerID="cri-o://3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a" gracePeriod=120
Jul 08 06:57:27.752004 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.751757    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="config-reloader" containerID="cri-o://4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9" gracePeriod=120
Jul 08 06:57:27.752004 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.751784    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy" containerID="cri-o://e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8" gracePeriod=120
Jul 08 06:57:27.752004 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.751762    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-web" containerID="cri-o://6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c" gracePeriod=120
Jul 08 06:57:27.752004 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:27.751964    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="prom-label-proxy" containerID="cri-o://1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932" gracePeriod=120
Jul 08 06:57:28.758592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758558    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932" exitCode=0
Jul 08 06:57:28.758592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758584    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8" exitCode=0
Jul 08 06:57:28.758592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758590    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9" exitCode=0
Jul 08 06:57:28.758592 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758596    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7" exitCode=0
Jul 08 06:57:28.759103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758625    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"}
Jul 08 06:57:28.759103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758729    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"}
Jul 08 06:57:28.759103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758759    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"}
Jul 08 06:57:28.759103 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:28.758783    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"}
Jul 08 06:57:29.008274 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.008249    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.030063 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.029991    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030063 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030041    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030248 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030076    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-volume\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030248 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030106    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-trusted-ca-bundle\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030248 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030152    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-cluster-tls-config\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030248 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030179    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-out\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030248 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030202    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-tls-assets\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030248 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030230    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-web-config\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030538 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030260    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-main-tls\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030538 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030298    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-web\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030538 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030326    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-metrics-client-ca\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030538 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030377    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jv5g\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-kube-api-access-7jv5g\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030538 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030409    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-main-db\") pod \"58d03695-168e-40bf-a0d4-1be35ca18cd6\" (UID: \"58d03695-168e-40bf-a0d4-1be35ca18cd6\") "
Jul 08 06:57:29.030789 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030562    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:29.030789 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.030645    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.031031 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.031008    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 06:57:29.031417 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.031386    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:29.033952 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.033771    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.033952 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.033878    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.033952 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.033898    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.034756 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.034718    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-out" (OuterVolumeSpecName: "config-out") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 06:57:29.034862 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.034793    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:57:29.034943 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.034905    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.035194 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.035173    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-volume" (OuterVolumeSpecName: "config-volume") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.036402 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.036374    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-kube-api-access-7jv5g" (OuterVolumeSpecName: "kube-api-access-7jv5g") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "kube-api-access-7jv5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:57:29.039680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.039657    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.048223 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.048195    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-web-config" (OuterVolumeSpecName: "web-config") pod "58d03695-168e-40bf-a0d4-1be35ca18cd6" (UID: "58d03695-168e-40bf-a0d4-1be35ca18cd6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:29.131447 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131413    2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-cluster-tls-config\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131447 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131440    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-out\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131447 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131452    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-tls-assets\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131464    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-web-config\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131477    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-main-tls\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131491    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131503    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58d03695-168e-40bf-a0d4-1be35ca18cd6-metrics-client-ca\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131514    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jv5g\" (UniqueName: \"kubernetes.io/projected/58d03695-168e-40bf-a0d4-1be35ca18cd6-kube-api-access-7jv5g\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131525    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58d03695-168e-40bf-a0d4-1be35ca18cd6-alertmanager-main-db\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131538    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131549    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.131679 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.131564    2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58d03695-168e-40bf-a0d4-1be35ca18cd6-config-volume\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:29.765528 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765490    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a" exitCode=0
Jul 08 06:57:29.765528 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765522    2571 generic.go:356] "Generic (PLEG): container finished" podID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerID="6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c" exitCode=0
Jul 08 06:57:29.765962 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765582    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.765962 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"}
Jul 08 06:57:29.765962 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765726    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"}
Jul 08 06:57:29.765962 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765757    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58d03695-168e-40bf-a0d4-1be35ca18cd6","Type":"ContainerDied","Data":"8d36d715bf5cb0b602c5e53971544ae31fde0ed5e12cb55f8b60e9ee5b1b6d93"}
Jul 08 06:57:29.765962 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.765797    2571 scope.go:117] "RemoveContainer" containerID="1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"
Jul 08 06:57:29.774522 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.774502    2571 scope.go:117] "RemoveContainer" containerID="3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"
Jul 08 06:57:29.781912 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.781895    2571 scope.go:117] "RemoveContainer" containerID="e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"
Jul 08 06:57:29.789353 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.789327    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:57:29.789769 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.789755    2571 scope.go:117] "RemoveContainer" containerID="6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"
Jul 08 06:57:29.795383 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.795358    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:57:29.797310 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.797292    2571 scope.go:117] "RemoveContainer" containerID="4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"
Jul 08 06:57:29.804345 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.804323    2571 scope.go:117] "RemoveContainer" containerID="16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"
Jul 08 06:57:29.811308 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.811290    2571 scope.go:117] "RemoveContainer" containerID="aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"
Jul 08 06:57:29.818743 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.818724    2571 scope.go:117] "RemoveContainer" containerID="1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"
Jul 08 06:57:29.819043 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.819015    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932\": container with ID starting with 1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932 not found: ID does not exist" containerID="1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"
Jul 08 06:57:29.819126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.819055    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"} err="failed to get container status \"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932\": rpc error: code = NotFound desc = could not find container \"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932\": container with ID starting with 1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932 not found: ID does not exist"
Jul 08 06:57:29.819126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.819079    2571 scope.go:117] "RemoveContainer" containerID="3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"
Jul 08 06:57:29.819369 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.819346    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a\": container with ID starting with 3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a not found: ID does not exist" containerID="3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"
Jul 08 06:57:29.819424 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.819377    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"} err="failed to get container status \"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a\": rpc error: code = NotFound desc = could not find container \"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a\": container with ID starting with 3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a not found: ID does not exist"
Jul 08 06:57:29.819424 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.819399    2571 scope.go:117] "RemoveContainer" containerID="e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"
Jul 08 06:57:29.819823 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.819777    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8\": container with ID starting with e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8 not found: ID does not exist" containerID="e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"
Jul 08 06:57:29.819823 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.819812    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"} err="failed to get container status \"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8\": rpc error: code = NotFound desc = could not find container \"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8\": container with ID starting with e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8 not found: ID does not exist"
Jul 08 06:57:29.820390 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.819835    2571 scope.go:117] "RemoveContainer" containerID="6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"
Jul 08 06:57:29.820390 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.820196    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c\": container with ID starting with 6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c not found: ID does not exist" containerID="6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"
Jul 08 06:57:29.820390 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.820222    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"} err="failed to get container status \"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c\": rpc error: code = NotFound desc = could not find container \"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c\": container with ID starting with 6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c not found: ID does not exist"
Jul 08 06:57:29.820390 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.820240    2571 scope.go:117] "RemoveContainer" containerID="4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"
Jul 08 06:57:29.821047 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.820939    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9\": container with ID starting with 4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9 not found: ID does not exist" containerID="4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"
Jul 08 06:57:29.821047 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.820981    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"} err="failed to get container status \"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9\": rpc error: code = NotFound desc = could not find container \"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9\": container with ID starting with 4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9 not found: ID does not exist"
Jul 08 06:57:29.821047 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.821001    2571 scope.go:117] "RemoveContainer" containerID="16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"
Jul 08 06:57:29.821388 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.821282    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7\": container with ID starting with 16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7 not found: ID does not exist" containerID="16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"
Jul 08 06:57:29.821388 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.821309    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"} err="failed to get container status \"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7\": rpc error: code = NotFound desc = could not find container \"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7\": container with ID starting with 16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7 not found: ID does not exist"
Jul 08 06:57:29.821388 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.821329    2571 scope.go:117] "RemoveContainer" containerID="aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"
Jul 08 06:57:29.821730 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:29.821677    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74\": container with ID starting with aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74 not found: ID does not exist" containerID="aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"
Jul 08 06:57:29.821818 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.821740    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"} err="failed to get container status \"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74\": rpc error: code = NotFound desc = could not find container \"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74\": container with ID starting with aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74 not found: ID does not exist"
Jul 08 06:57:29.821818 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.821762    2571 scope.go:117] "RemoveContainer" containerID="1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"
Jul 08 06:57:29.821969 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.821909    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:57:29.822062 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822031    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932"} err="failed to get container status \"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932\": rpc error: code = NotFound desc = could not find container \"1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932\": container with ID starting with 1f91aa1ccabf1c95e0a5092d8535ca50245b96964de724c9c4a15308425ae932 not found: ID does not exist"
Jul 08 06:57:29.822272 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822063    2571 scope.go:117] "RemoveContainer" containerID="3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822277    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-metric"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822294    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-metric"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822308    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="alertmanager"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822318    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="alertmanager"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822330    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e6a281d-ff8d-450c-b252-e9b0c319a152" containerName="console"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822339    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e6a281d-ff8d-450c-b252-e9b0c319a152" containerName="console"
Jul 08 06:57:29.822352 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822329    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a"} err="failed to get container status \"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a\": rpc error: code = NotFound desc = could not find container \"3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a\": container with ID starting with 3c7e651fda1abdcb072af532de9521a59e7090edaca1f06f78d0d12a7489830a not found: ID does not exist"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822355    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="init-config-reloader"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822359    2571 scope.go:117] "RemoveContainer" containerID="e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822364    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="init-config-reloader"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822375    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="config-reloader"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822385    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="config-reloader"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822400    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="prom-label-proxy"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822408    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="prom-label-proxy"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822418    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-web"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822427    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-web"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822437    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822447    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822469    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4077e488-ca01-4396-9f45-cebd5fbdcef1" containerName="registry"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822477    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4077e488-ca01-4396-9f45-cebd5fbdcef1" containerName="registry"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822545    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="alertmanager"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822556    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-web"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822568    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy-metric"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822576    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="prom-label-proxy"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822587    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="config-reloader"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822596    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4077e488-ca01-4396-9f45-cebd5fbdcef1" containerName="registry"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822591    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8"} err="failed to get container status \"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8\": rpc error: code = NotFound desc = could not find container \"e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8\": container with ID starting with e450bc87f7318d6a19928d22a1a0b5e674d7f396c5c2d54be91aeeb4d83ff8b8 not found: ID does not exist"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822605    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e6a281d-ff8d-450c-b252-e9b0c319a152" containerName="console"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822613    2571 scope.go:117] "RemoveContainer" containerID="6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"
Jul 08 06:57:29.822675 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.822616    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" containerName="kube-rbac-proxy"
Jul 08 06:57:29.823590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823039    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c"} err="failed to get container status \"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c\": rpc error: code = NotFound desc = could not find container \"6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c\": container with ID starting with 6881e81ddfff41c9cf9107646d5f1442a2254465f8efb5bcf3e36632972a8f8c not found: ID does not exist"
Jul 08 06:57:29.823590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823063    2571 scope.go:117] "RemoveContainer" containerID="4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"
Jul 08 06:57:29.823590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823316    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9"} err="failed to get container status \"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9\": rpc error: code = NotFound desc = could not find container \"4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9\": container with ID starting with 4dcc5d610df734e8370a503e352bd4f2a5e81637a043999706764a3ccb709fd9 not found: ID does not exist"
Jul 08 06:57:29.823590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823337    2571 scope.go:117] "RemoveContainer" containerID="16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"
Jul 08 06:57:29.823590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823559    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7"} err="failed to get container status \"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7\": rpc error: code = NotFound desc = could not find container \"16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7\": container with ID starting with 16dc4e5cec37b72accae2bbd97b3c054e3c1f562a34df51807a0543610b442a7 not found: ID does not exist"
Jul 08 06:57:29.823590 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823578    2571 scope.go:117] "RemoveContainer" containerID="aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"
Jul 08 06:57:29.823852 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.823810    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74"} err="failed to get container status \"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74\": rpc error: code = NotFound desc = could not find container \"aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74\": container with ID starting with aebf4215bb8fe66494dd0750ce40bd9da3b3af7a47f2684948a13374e47faf74 not found: ID does not exist"
Jul 08 06:57:29.828375 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.828356    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.831006 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.830986    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 08 06:57:29.831081 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.830986    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 08 06:57:29.831636 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831614    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 08 06:57:29.831636 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831636    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 08 06:57:29.831802 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831668    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 08 06:57:29.831802 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831708    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 08 06:57:29.831802 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831739    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 08 06:57:29.831802 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831753    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 08 06:57:29.832010 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.831995    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7tg6l\""
Jul 08 06:57:29.836167 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.836121    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 08 06:57:29.837004 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.836959    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/348be2f3-97e5-4013-b12b-5db73dff5b1a-config-out\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837004 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.836986    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837048    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/348be2f3-97e5-4013-b12b-5db73dff5b1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837109    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348be2f3-97e5-4013-b12b-5db73dff5b1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837172    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/348be2f3-97e5-4013-b12b-5db73dff5b1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/348be2f3-97e5-4013-b12b-5db73dff5b1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837286    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c92zx\" (UniqueName: \"kubernetes.io/projected/348be2f3-97e5-4013-b12b-5db73dff5b1a-kube-api-access-c92zx\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837327    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-web-config\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.837548 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.837412    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.840054 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.840027    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:57:29.938364 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/348be2f3-97e5-4013-b12b-5db73dff5b1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938532 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938378    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c92zx\" (UniqueName: \"kubernetes.io/projected/348be2f3-97e5-4013-b12b-5db73dff5b1a-kube-api-access-c92zx\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938532 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938650 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938529    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-web-config\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938650 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938791 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938725    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938791 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/348be2f3-97e5-4013-b12b-5db73dff5b1a-config-out\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938896 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938808    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938896 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938864    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938998 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938898    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/348be2f3-97e5-4013-b12b-5db73dff5b1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938998 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938931    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348be2f3-97e5-4013-b12b-5db73dff5b1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.938998 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.938968    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.939158 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.939008    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/348be2f3-97e5-4013-b12b-5db73dff5b1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.939557 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.939223    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/348be2f3-97e5-4013-b12b-5db73dff5b1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.939842 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.939667    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/348be2f3-97e5-4013-b12b-5db73dff5b1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.940467 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.940438    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/348be2f3-97e5-4013-b12b-5db73dff5b1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.941714 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.941672    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.941806 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.941719    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.942005 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.941983    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/348be2f3-97e5-4013-b12b-5db73dff5b1a-config-out\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.942107 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.942029    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-web-config\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.942107 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.942031    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.942466 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.942448    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.942626 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.942610    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.943084 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.943064    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/348be2f3-97e5-4013-b12b-5db73dff5b1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.943777 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.943761    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/348be2f3-97e5-4013-b12b-5db73dff5b1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:29.951386 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:29.951367    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c92zx\" (UniqueName: \"kubernetes.io/projected/348be2f3-97e5-4013-b12b-5db73dff5b1a-kube-api-access-c92zx\") pod \"alertmanager-main-0\" (UID: \"348be2f3-97e5-4013-b12b-5db73dff5b1a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:30.138359 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:30.138332    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 06:57:30.288230 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:30.288199    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 06:57:30.289543 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:57:30.289500    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348be2f3_97e5_4013_b12b_5db73dff5b1a.slice/crio-1586435a47833cfd17eeae0be08ec22ee0db595d1030e12421ade590982097dd WatchSource:0}: Error finding container 1586435a47833cfd17eeae0be08ec22ee0db595d1030e12421ade590982097dd: Status 404 returned error can't find the container with id 1586435a47833cfd17eeae0be08ec22ee0db595d1030e12421ade590982097dd
Jul 08 06:57:30.770507 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:30.770470    2571 generic.go:356] "Generic (PLEG): container finished" podID="348be2f3-97e5-4013-b12b-5db73dff5b1a" containerID="0ce25962b56e5a9ce24d41ed1482641eb77174bb1b6e368a50ad841b11ce7de2" exitCode=0
Jul 08 06:57:30.770968 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:30.770562    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerDied","Data":"0ce25962b56e5a9ce24d41ed1482641eb77174bb1b6e368a50ad841b11ce7de2"}
Jul 08 06:57:30.770968 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:30.770645    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"1586435a47833cfd17eeae0be08ec22ee0db595d1030e12421ade590982097dd"}
Jul 08 06:57:30.913964 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:30.913933    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d03695-168e-40bf-a0d4-1be35ca18cd6" path="/var/lib/kubelet/pods/58d03695-168e-40bf-a0d4-1be35ca18cd6/volumes"
Jul 08 06:57:31.367537 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.365815    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-847cf488d5-2pjrx"]
Jul 08 06:57:31.370072 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.370041    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.373634 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.373378    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 08 06:57:31.373799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.373713    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 08 06:57:31.374095 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.374071    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 08 06:57:31.374216 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.374153    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 08 06:57:31.374216 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.374077    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 08 06:57:31.374317 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.374255    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-656n2\""
Jul 08 06:57:31.379169 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.379146    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 08 06:57:31.383918 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.383896    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-847cf488d5-2pjrx"]
Jul 08 06:57:31.450368 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450335    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-secret-telemeter-client\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450519 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450376    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-serving-certs-ca-bundle\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450519 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450402    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450519 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450464    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-federate-client-tls\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450519 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450502    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-telemeter-trusted-ca-bundle\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450731 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450527    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-metrics-client-ca\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450731 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450578    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-telemeter-client-tls\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.450731 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.450616    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjz8m\" (UniqueName: \"kubernetes.io/projected/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-kube-api-access-tjz8m\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551502 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551461    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-secret-telemeter-client\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-serving-certs-ca-bundle\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-federate-client-tls\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551652    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-telemeter-trusted-ca-bundle\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551711    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-metrics-client-ca\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551987 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551761    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-telemeter-client-tls\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.551987 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.551793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjz8m\" (UniqueName: \"kubernetes.io/projected/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-kube-api-access-tjz8m\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.553521 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.553487    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-serving-certs-ca-bundle\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.553712 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.553667    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-metrics-client-ca\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.553890 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.553861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-telemeter-trusted-ca-bundle\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.555230 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.555206    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-telemeter-client-tls\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.555337 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.555233    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-federate-client-tls\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.555399 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.555389    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-secret-telemeter-client\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.555452 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.555438    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.565303 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.565277    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjz8m\" (UniqueName: \"kubernetes.io/projected/61878bb7-e686-4acb-b1f5-a9b0b6c686ff-kube-api-access-tjz8m\") pod \"telemeter-client-847cf488d5-2pjrx\" (UID: \"61878bb7-e686-4acb-b1f5-a9b0b6c686ff\") " pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.674365 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674269    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:57:31.674799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674551    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="prometheus" containerID="cri-o://88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f" gracePeriod=600
Jul 08 06:57:31.674799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674569    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy" containerID="cri-o://9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558" gracePeriod=600
Jul 08 06:57:31.674799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674588    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="thanos-sidecar" containerID="cri-o://26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c" gracePeriod=600
Jul 08 06:57:31.674799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674602    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="config-reloader" containerID="cri-o://6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a" gracePeriod=600
Jul 08 06:57:31.674799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674634    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-web" containerID="cri-o://6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a" gracePeriod=600
Jul 08 06:57:31.674799 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.674726    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f" gracePeriod=600
Jul 08 06:57:31.683802 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.683648    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx"
Jul 08 06:57:31.782198 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.782153    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"d27920186533d0f229592c1396f361ab5669ab15bf8024508f585833cfddf71a"}
Jul 08 06:57:31.782646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.782244    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"96cfa82ae5533b1885bfa7a2ceb0f6d9b478cb644f604754c57a3c7ce77df35f"}
Jul 08 06:57:31.782646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.782287    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"933657dc149388e894f1e9b84d2f3a1bb46209b690880afa6acef8a95089ed25"}
Jul 08 06:57:31.782646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.782323    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"373118840bc38c2a716633acd2842ec5466d201e7f73aa9e9c84e10f06e4eb0c"}
Jul 08 06:57:31.782646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.782362    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"fbbd185840b177bcf8ff1e92022bd4ab8e2f458e2df6b228f2e80029cff50015"}
Jul 08 06:57:31.782646 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.782400    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"348be2f3-97e5-4013-b12b-5db73dff5b1a","Type":"ContainerStarted","Data":"c0e53fec13be8460e3c54a8ef72156bb6cc0c2f28003b5967bfc890af803193f"}
Jul 08 06:57:31.842909 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.838947    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.821984453 podStartE2EDuration="2.838913637s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.929184ms" isStatefulPod=false podCreationTimestamp="2026-07-08 06:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:57:31.829375857 +0000 UTC m=+179.428665560" watchObservedRunningTime="2026-07-08 06:57:31.838913637 +0000 UTC m=+179.438203341"
Jul 08 06:57:31.888446 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:31.888421    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-847cf488d5-2pjrx"]
Jul 08 06:57:31.891237 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:57:31.891206    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61878bb7_e686_4acb_b1f5_a9b0b6c686ff.slice/crio-54c876362d83ded09688723ed5e1cbd750bdbda1d5f90da1f5e09927b78aa332 WatchSource:0}: Error finding container 54c876362d83ded09688723ed5e1cbd750bdbda1d5f90da1f5e09927b78aa332: Status 404 returned error can't find the container with id 54c876362d83ded09688723ed5e1cbd750bdbda1d5f90da1f5e09927b78aa332
Jul 08 06:57:32.009911 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.009886    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.058719 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.058665    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-serving-certs-ca-bundle\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.058954 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.058932    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-tls\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.059125 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059080    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.059255 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059137    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-trusted-ca-bundle\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.059255 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059212    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-metrics-client-ca\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.059476 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059440    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.059772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059539    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.059772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059601    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-kube-rbac-proxy\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.059772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059628    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-grpc-tls\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.059772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059718    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-metrics-client-certs\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.059772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059750    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059787    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-2\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059813    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-kubelet-serving-ca-bundle\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059840    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059865    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-config-out\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059896    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-1\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059918    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-thanos-prometheus-http-client-file\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059960    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-db\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.059991    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-config\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060034    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-web-config\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060067    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-tls-assets\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060097    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdf5w\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-kube-api-access-gdf5w\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.060443 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060121    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-0\") pod \"2f248415-02b3-452d-8e0f-b0ca32051a27\" (UID: \"2f248415-02b3-452d-8e0f-b0ca32051a27\") "
Jul 08 06:57:32.061173 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060721    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.061173 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060748    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.061173 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.060765    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-metrics-client-ca\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.061173 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.061134    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.061459 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.061432    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.062019 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.061745    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.062019 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.061789    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.063137 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.063020    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 06:57:32.064918 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.064893    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 06:57:32.065724 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.065660    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:57:32.066863 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.066822    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.066962 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.066857    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-kube-api-access-gdf5w" (OuterVolumeSpecName: "kube-api-access-gdf5w") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "kube-api-access-gdf5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 06:57:32.067036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.067007    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.067100 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.067028    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.067292 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.067220    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-config-out" (OuterVolumeSpecName: "config-out") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 06:57:32.067292 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.067219    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-config" (OuterVolumeSpecName: "config") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.067855 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.067822    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.068330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.068304    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.068912 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.068889    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.077420 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.077399    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-web-config" (OuterVolumeSpecName: "web-config") pod "2f248415-02b3-452d-8e0f-b0ca32051a27" (UID: "2f248415-02b3-452d-8e0f-b0ca32051a27"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 06:57:32.161729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161659    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-web-config\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.161729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161701    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-tls-assets\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.161729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161711    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdf5w\" (UniqueName: \"kubernetes.io/projected/2f248415-02b3-452d-8e0f-b0ca32051a27-kube-api-access-gdf5w\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.161729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161722    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.161729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161732    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.161729 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161741    2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-kube-rbac-proxy\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161749    2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-grpc-tls\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161757    2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-metrics-client-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161767    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161775    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161783    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161792    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161800    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-config-out\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161808    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161816    2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161823    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f248415-02b3-452d-8e0f-b0ca32051a27-prometheus-k8s-db\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.162035 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.161831    2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f248415-02b3-452d-8e0f-b0ca32051a27-config\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 06:57:32.790264 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790226    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f" exitCode=0
Jul 08 06:57:32.790264 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790255    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558" exitCode=0
Jul 08 06:57:32.790264 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790264    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a" exitCode=0
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790274    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c" exitCode=0
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790283    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a" exitCode=0
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790291    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f" exitCode=0
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790303    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"}
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790319    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"}
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790398    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"}
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790440    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"}
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"}
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790442    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790520    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"}
Jul 08 06:57:32.790849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.790557    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f248415-02b3-452d-8e0f-b0ca32051a27","Type":"ContainerDied","Data":"fe06b5dd8f5ae4c6bbd5e14089c550fc5f2f7925c2c0f5dfea163b8268723c32"}
Jul 08 06:57:32.792107 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.792027    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx" event={"ID":"61878bb7-e686-4acb-b1f5-a9b0b6c686ff","Type":"ContainerStarted","Data":"54c876362d83ded09688723ed5e1cbd750bdbda1d5f90da1f5e09927b78aa332"}
Jul 08 06:57:32.800985 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.800962    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.810518 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.810490    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.817653 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.817629    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:57:32.822273 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.822251    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:57:32.828644 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.828527    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.832605 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.832577    2571 kuberuntime_gc.go:392] "Failed to remove container log dead symlink" err="remove /var/log/containers/prometheus-k8s-0_openshift-monitoring_thanos-sidecar-26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c.log: no such file or directory" path="/var/log/containers/prometheus-k8s-0_openshift-monitoring_thanos-sidecar-26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c.log"
Jul 08 06:57:32.838025 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.838009    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.846281 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.846190    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.848446 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848426    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:57:32.848807 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848792    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="prometheus"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848809    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="prometheus"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848820    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="config-reloader"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848828    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="config-reloader"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848838    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-thanos"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848846    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-thanos"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848888    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="thanos-sidecar"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848896    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="thanos-sidecar"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848912    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-web"
Jul 08 06:57:32.848924 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848922    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-web"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848934    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="init-config-reloader"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848942    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="init-config-reloader"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848956    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.848964    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.849037    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="thanos-sidecar"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.849050    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-thanos"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.849060    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.849069    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="prometheus"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.849080    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="kube-rbac-proxy-web"
Jul 08 06:57:32.849224 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.849089    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" containerName="config-reloader"
Jul 08 06:57:32.854292 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.854273    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.855488 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.855467    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.857188 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.857168    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8scvq\""
Jul 08 06:57:32.857753 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.857733    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 06:57:32.857895 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.857776    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 06:57:32.858046 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.858030    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 06:57:32.858249 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.858233    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.858651    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.858872    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.859202    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.859672    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.859970    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.860058    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.860161    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-n77s5p8qvh3b\""
Jul 08 06:57:32.860330 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.860209    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 06:57:32.861198 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.860339    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 06:57:32.861529 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.861509    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 06:57:32.864730 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.864708    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868402    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868780    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868833    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6drt\" (UniqueName: \"kubernetes.io/projected/a5c269d5-c208-4018-862d-cf99e46dea23-kube-api-access-z6drt\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868912    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868968    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.868996    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869041    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-web-config\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5c269d5-c208-4018-862d-cf99e46dea23-config-out\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869203    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869279    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869335    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5c269d5-c208-4018-862d-cf99e46dea23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.869900 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869355    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.870711 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869385    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.870711 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869411    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-config\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.870711 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.870711 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.869490    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.871415 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.871390    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.871484 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.871427    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"} err="failed to get container status \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist"
Jul 08 06:57:32.871484 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.871464    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.871767 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.871745    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.871862 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.871776    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"} err="failed to get container status \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist"
Jul 08 06:57:32.871862 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.871796    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.872933 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.872915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:57:32.873105 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.873077    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.873160 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.873115    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"} err="failed to get container status \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist"
Jul 08 06:57:32.873160 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.873139    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.873535 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.873510    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.873614 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.873545    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"} err="failed to get container status \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist"
Jul 08 06:57:32.873614 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.873569    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.873910 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.873893    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.873984 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.873916    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"} err="failed to get container status \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist"
Jul 08 06:57:32.873984 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.873930    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.874195 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.874167    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.874291 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.874199    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"} err="failed to get container status \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist"
Jul 08 06:57:32.874291 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.874220    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.874520 ip-10-0-129-54 kubenswrapper[2571]: E0708 06:57:32.874484    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.874593 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.874531    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"} err="failed to get container status \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist"
Jul 08 06:57:32.874593 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.874555    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.874854 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.874829    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"} err="failed to get container status \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist"
Jul 08 06:57:32.874946 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.874856    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.875407 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.875374    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"} err="failed to get container status \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist"
Jul 08 06:57:32.875527 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.875507    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.875933 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.875906    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"} err="failed to get container status \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist"
Jul 08 06:57:32.876039 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.875934    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.876243 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.876221    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"} err="failed to get container status \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist"
Jul 08 06:57:32.876340 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.876245    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.876639 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.876553    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"} err="failed to get container status \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist"
Jul 08 06:57:32.876639 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.876576    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.877032 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877002    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"} err="failed to get container status \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist"
Jul 08 06:57:32.877032 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877032    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.877290 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877270    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"} err="failed to get container status \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist"
Jul 08 06:57:32.877290 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877290    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.877502 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877484    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"} err="failed to get container status \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist"
Jul 08 06:57:32.877502 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877502    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.877772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877746    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"} err="failed to get container status \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist"
Jul 08 06:57:32.877772 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.877770    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.878036 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878008    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"} err="failed to get container status \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist"
Jul 08 06:57:32.878105 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878040    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.878329 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878287    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"} err="failed to get container status \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist"
Jul 08 06:57:32.878414 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878332    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.878637 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878610    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"} err="failed to get container status \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist"
Jul 08 06:57:32.878740 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878640    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.878909 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878891    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"} err="failed to get container status \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist"
Jul 08 06:57:32.878993 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.878911    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.879163 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.879138    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"} err="failed to get container status \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist"
Jul 08 06:57:32.879216 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.879165    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.879406 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.879381    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"} err="failed to get container status \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist"
Jul 08 06:57:32.879471 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.879409    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.879722 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.879674    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"} err="failed to get container status \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist"
Jul 08 06:57:32.879798 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.879722    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.880024 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880003    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"} err="failed to get container status \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist"
Jul 08 06:57:32.880096 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880025    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.880282 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880255    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"} err="failed to get container status \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist"
Jul 08 06:57:32.880377 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880283    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.880529 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880511    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"} err="failed to get container status \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist"
Jul 08 06:57:32.880529 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880528    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.880785 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880757    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"} err="failed to get container status \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist"
Jul 08 06:57:32.880785 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.880785    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.881175 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.881154    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"} err="failed to get container status \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist"
Jul 08 06:57:32.881175 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.881171    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.881395 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.881371    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"} err="failed to get container status \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist"
Jul 08 06:57:32.881395 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.881395    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.881736 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.881679    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"} err="failed to get container status \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist"
Jul 08 06:57:32.881838 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.881737    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.882039 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882006    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"} err="failed to get container status \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist"
Jul 08 06:57:32.882132 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882049    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.882313 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882293    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"} err="failed to get container status \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist"
Jul 08 06:57:32.882313 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882314    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.882653 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882628    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"} err="failed to get container status \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist"
Jul 08 06:57:32.882754 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882654    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.883068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.882992    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"} err="failed to get container status \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist"
Jul 08 06:57:32.883068 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883013    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.883278 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883258    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"} err="failed to get container status \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist"
Jul 08 06:57:32.883278 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883278    2571 scope.go:117] "RemoveContainer" containerID="6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"
Jul 08 06:57:32.883516 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883498    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f"} err="failed to get container status \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": rpc error: code = NotFound desc = could not find container \"6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f\": container with ID starting with 6b13c466ac6cb3af671b17c782da4f812970b16e01c3edc21f3e652e1257052f not found: ID does not exist"
Jul 08 06:57:32.883516 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883516    2571 scope.go:117] "RemoveContainer" containerID="9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"
Jul 08 06:57:32.883781 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883762    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558"} err="failed to get container status \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": rpc error: code = NotFound desc = could not find container \"9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558\": container with ID starting with 9cdd0a79a8716902b7b110af3f7e34779c216018bf8f94c8a926c87741fc5558 not found: ID does not exist"
Jul 08 06:57:32.883849 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.883781    2571 scope.go:117] "RemoveContainer" containerID="6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"
Jul 08 06:57:32.884043 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884023    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a"} err="failed to get container status \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": rpc error: code = NotFound desc = could not find container \"6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a\": container with ID starting with 6d15ba3977cf82744305b299d3fc9e86742e81d08554ec6863d345f2afef925a not found: ID does not exist"
Jul 08 06:57:32.884110 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884046    2571 scope.go:117] "RemoveContainer" containerID="26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"
Jul 08 06:57:32.884316 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884300    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c"} err="failed to get container status \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": rpc error: code = NotFound desc = could not find container \"26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c\": container with ID starting with 26ba268b22109f5a42b57241aad148672c0f0e0b6e8f5aa37ddf76c91a5eed2c not found: ID does not exist"
Jul 08 06:57:32.884378 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884316    2571 scope.go:117] "RemoveContainer" containerID="6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"
Jul 08 06:57:32.884579 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884552    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a"} err="failed to get container status \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": rpc error: code = NotFound desc = could not find container \"6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a\": container with ID starting with 6225d7df9cbfb5cb0d2d86885681633d8e6663a4a8ef6f11ee5429e65c35237a not found: ID does not exist"
Jul 08 06:57:32.884680 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884580    2571 scope.go:117] "RemoveContainer" containerID="88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"
Jul 08 06:57:32.884859 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884833    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f"} err="failed to get container status \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": rpc error: code = NotFound desc = could not find container \"88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f\": container with ID starting with 88aac62e5635f1becf553e7567e54b7e07481b890aed567f7c472d1c446b339f not found: ID does not exist"
Jul 08 06:57:32.884926 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.884860    2571 scope.go:117] "RemoveContainer" containerID="ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"
Jul 08 06:57:32.885109 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.885087    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794"} err="failed to get container status \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": rpc error: code = NotFound desc = could not find container \"ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794\": container with ID starting with ede24cadff67fc7dcfe415f1a8b401eb759c68724ab3ebb79dd687ae75a15794 not found: ID does not exist"
Jul 08 06:57:32.919612 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.919579    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f248415-02b3-452d-8e0f-b0ca32051a27" path="/var/lib/kubelet/pods/2f248415-02b3-452d-8e0f-b0ca32051a27/volumes"
Jul 08 06:57:32.970478 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970445    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6drt\" (UniqueName: \"kubernetes.io/projected/a5c269d5-c208-4018-862d-cf99e46dea23-kube-api-access-z6drt\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970654 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970488    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970654 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970529    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970654 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970548    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970654 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970654 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-web-config\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970654 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970649    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970676    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5c269d5-c208-4018-862d-cf99e46dea23-config-out\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970729    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970761    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970789    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970814    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970849    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5c269d5-c208-4018-862d-cf99e46dea23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970948    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.970996 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.970977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-config\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.971487 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.971007    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.971487 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.971035    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.971487 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.971068    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.971650 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.971597    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.971751 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.971729    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.971821 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.971737    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.972393 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.972364    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.972490 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.972453    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.972953 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.972927    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.974985 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.974957    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.975505 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.975481    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a5c269d5-c208-4018-862d-cf99e46dea23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.975894 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.975870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5c269d5-c208-4018-862d-cf99e46dea23-config-out\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.975997 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.975901    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.976770 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.976734    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.976862 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.976847    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.977098 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.977071    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5c269d5-c208-4018-862d-cf99e46dea23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.977463 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.977441    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-web-config\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.978031 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.978007    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.978455 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.978431    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.978940 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.978902    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.979136 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.979115    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.980182 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.980159    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a5c269d5-c208-4018-862d-cf99e46dea23-config\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:32.980397 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:32.980376    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6drt\" (UniqueName: \"kubernetes.io/projected/a5c269d5-c208-4018-862d-cf99e46dea23-kube-api-access-z6drt\") pod \"prometheus-k8s-0\" (UID: \"a5c269d5-c208-4018-862d-cf99e46dea23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:33.168858 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:33.168821    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:57:33.327250 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:33.327215    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 06:57:33.719467 ip-10-0-129-54 kubenswrapper[2571]: W0708 06:57:33.719430    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c269d5_c208_4018_862d_cf99e46dea23.slice/crio-4a25cc147c17c88db15cf2e3400c9b669cb25e780fd190e8806cb8c68d5c53ec WatchSource:0}: Error finding container 4a25cc147c17c88db15cf2e3400c9b669cb25e780fd190e8806cb8c68d5c53ec: Status 404 returned error can't find the container with id 4a25cc147c17c88db15cf2e3400c9b669cb25e780fd190e8806cb8c68d5c53ec
Jul 08 06:57:33.797319 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:33.797293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"613300b3c7ea4d20f84deea816ceb5351a6d84b76b5e85f17eb792879827b5fd"}
Jul 08 06:57:33.797610 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:33.797355    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"4a25cc147c17c88db15cf2e3400c9b669cb25e780fd190e8806cb8c68d5c53ec"}
Jul 08 06:57:34.802597 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:34.802556    2571 generic.go:356] "Generic (PLEG): container finished" podID="a5c269d5-c208-4018-862d-cf99e46dea23" containerID="613300b3c7ea4d20f84deea816ceb5351a6d84b76b5e85f17eb792879827b5fd" exitCode=0
Jul 08 06:57:34.803075 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:34.802651    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerDied","Data":"613300b3c7ea4d20f84deea816ceb5351a6d84b76b5e85f17eb792879827b5fd"}
Jul 08 06:57:34.805089 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:34.805054    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx" event={"ID":"61878bb7-e686-4acb-b1f5-a9b0b6c686ff","Type":"ContainerStarted","Data":"72495eff47fc100e100f75e13196ae3729a6f4124479fb4683ad44aa3615ccfc"}
Jul 08 06:57:34.805189 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:34.805120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx" event={"ID":"61878bb7-e686-4acb-b1f5-a9b0b6c686ff","Type":"ContainerStarted","Data":"2b9a406636cda45435a7df7977fa1f595ac6a9a90776c5645bb94cc24d1812ec"}
Jul 08 06:57:34.805189 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:34.805151    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx" event={"ID":"61878bb7-e686-4acb-b1f5-a9b0b6c686ff","Type":"ContainerStarted","Data":"94334fbfddea3ae88fd0c634a3dcb08f83b00343d29a6899194201efde0effbb"}
Jul 08 06:57:34.858121 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:34.858080    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-847cf488d5-2pjrx" podStartSLOduration=1.982021595 podStartE2EDuration="3.858064807s" totalImagesPullingTime="1.876043212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 06:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:57:34.856478712 +0000 UTC m=+182.455768437" watchObservedRunningTime="2026-07-08 06:57:34.858064807 +0000 UTC m=+182.457354510"
Jul 08 06:57:35.814625 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.814584    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"685d85e900d19132d18ae1ab03c79b7ab38538ed35e70b3aca5bfc46b1c251fc"}
Jul 08 06:57:35.815126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.814716    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"5c563e8079b6d9476c4c6607b54dab0bb3fb9c324aedac6b4a6d5d7dcae0d8e9"}
Jul 08 06:57:35.815126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.814762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"4dcbaa4ff0ff0b279c17230d5137603a0ad466eaaca10c1b0c84b327971bddff"}
Jul 08 06:57:35.815126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.814810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"aad526360cdff6a7aedb5109fdb2a4e2f6eea7d558b95fef7d09ca7ddabdd5f1"}
Jul 08 06:57:35.815126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.814847    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"8591516c1d7a33fb34a5a153dca5c143c98a9f479b5151face0192fb8ebcf292"}
Jul 08 06:57:35.815126 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.814887    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a5c269d5-c208-4018-862d-cf99e46dea23","Type":"ContainerStarted","Data":"7246a3775640ca003753fbd8d3eef3f3b19f2e394a1819b69d1f85d653f42106"}
Jul 08 06:57:35.851228 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:35.851169    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.826122201 podStartE2EDuration="3.851152016s" totalImagesPullingTime="0s" totalInitContainerRuntime="25.029815ms" isStatefulPod=false podCreationTimestamp="2026-07-08 06:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 06:57:35.848755561 +0000 UTC m=+183.448045264" watchObservedRunningTime="2026-07-08 06:57:35.851152016 +0000 UTC m=+183.450441718"
Jul 08 06:57:38.168985 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:57:38.168948    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:58:33.169022 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:58:33.168992    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:58:33.185183 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:58:33.185157    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:58:34.061798 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:58:34.061759    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 06:59:32.825455 ip-10-0-129-54 kubenswrapper[2571]: I0708 06:59:32.825420    2571 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 07:02:35.629326 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.629295    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j2hff"]
Jul 08 07:02:35.632492 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.632476    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.635590 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.635570    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-78w7t\""
Jul 08 07:02:35.635710 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.635634    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 08 07:02:35.635780 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.635738    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 08 07:02:35.641005 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.640982    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j2hff"]
Jul 08 07:02:35.736312 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.736276    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j2hff\" (UID: \"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.736483 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.736351    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlxrd\" (UniqueName: \"kubernetes.io/projected/ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a-kube-api-access-nlxrd\") pod \"cert-manager-webhook-6486ffd997-j2hff\" (UID: \"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.837744 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.837712    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxrd\" (UniqueName: \"kubernetes.io/projected/ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a-kube-api-access-nlxrd\") pod \"cert-manager-webhook-6486ffd997-j2hff\" (UID: \"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.837877 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.837777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j2hff\" (UID: \"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.856372 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.856348    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxrd\" (UniqueName: \"kubernetes.io/projected/ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a-kube-api-access-nlxrd\") pod \"cert-manager-webhook-6486ffd997-j2hff\" (UID: \"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.865210 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.865187    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j2hff\" (UID: \"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:35.942283 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:35.942214    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:36.078989 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:36.078967    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j2hff"]
Jul 08 07:02:36.081615 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:02:36.081588    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae12a5f3_9ea5_4ba0_bbe0_ee90460d207a.slice/crio-17ae12f8af33401e01b7adb30ec59548704051497eac8a0f7854caf94ca55afb WatchSource:0}: Error finding container 17ae12f8af33401e01b7adb30ec59548704051497eac8a0f7854caf94ca55afb: Status 404 returned error can't find the container with id 17ae12f8af33401e01b7adb30ec59548704051497eac8a0f7854caf94ca55afb
Jul 08 07:02:36.083606 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:36.083587    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 07:02:36.988424 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:36.988375    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff" event={"ID":"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a","Type":"ContainerStarted","Data":"17ae12f8af33401e01b7adb30ec59548704051497eac8a0f7854caf94ca55afb"}
Jul 08 07:02:40.003044 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:40.003008    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff" event={"ID":"ae12a5f3-9ea5-4ba0-bbe0-ee90460d207a","Type":"ContainerStarted","Data":"aa4d77278be6381335b3d365814ca261f96e720bc6deef9f5504127d515bfa06"}
Jul 08 07:02:40.003436 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:40.003066    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:40.020321 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:40.020277    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff" podStartSLOduration=1.747267963 podStartE2EDuration="5.02026127s" totalImagesPullingTime="3.272993307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:02:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:02:40.01835501 +0000 UTC m=+487.617644715" watchObservedRunningTime="2026-07-08 07:02:40.02026127 +0000 UTC m=+487.619550973"
Jul 08 07:02:41.330704 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.330651    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-765c4ccff4-242ps"]
Jul 08 07:02:41.333901 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.333884    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.336326 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.336301    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 07:02:41.337541 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.337520    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rg6rq\""
Jul 08 07:02:41.337541 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.337523    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 07:02:41.337752 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.337567    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 07:02:41.337752 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.337523    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 07:02:41.337874 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.337749    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 07:02:41.338083 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.338063    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 07:02:41.338216 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.338192    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 07:02:41.347150 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.347125    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 07:02:41.348572 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.348545    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765c4ccff4-242ps"]
Jul 08 07:02:41.489817 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.489778    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-oauth-config\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.489978 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.489840    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-config\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.489978 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.489903    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-service-ca\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.489978 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.489961    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-trusted-ca-bundle\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.490103 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.489999    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-oauth-serving-cert\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.490103 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.490026    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bb5g\" (UniqueName: \"kubernetes.io/projected/1c9142b1-8507-43d0-92a8-851fb514b3c4-kube-api-access-4bb5g\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.490103 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.490052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-serving-cert\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591118 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-config\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591118 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-service-ca\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591118 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-trusted-ca-bundle\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591349 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-oauth-serving-cert\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591349 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bb5g\" (UniqueName: \"kubernetes.io/projected/1c9142b1-8507-43d0-92a8-851fb514b3c4-kube-api-access-4bb5g\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591349 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591207    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-serving-cert\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591349 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-oauth-config\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.591902 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591873    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-config\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.592019 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591897    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-service-ca\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.592019 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.591947    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-oauth-serving-cert\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.592280 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.592256    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c9142b1-8507-43d0-92a8-851fb514b3c4-trusted-ca-bundle\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.594014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.593983    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-oauth-config\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.594014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.593992    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c9142b1-8507-43d0-92a8-851fb514b3c4-console-serving-cert\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.599494 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.599473    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bb5g\" (UniqueName: \"kubernetes.io/projected/1c9142b1-8507-43d0-92a8-851fb514b3c4-kube-api-access-4bb5g\") pod \"console-765c4ccff4-242ps\" (UID: \"1c9142b1-8507-43d0-92a8-851fb514b3c4\") " pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.643424 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.643390    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:41.780343 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:41.780314    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765c4ccff4-242ps"]
Jul 08 07:02:41.783322 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:02:41.783293    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9142b1_8507_43d0_92a8_851fb514b3c4.slice/crio-1f1eabb45902560a52f36b9e23786a983cb6eeb8da723e98e2fc31baf8d98fdf WatchSource:0}: Error finding container 1f1eabb45902560a52f36b9e23786a983cb6eeb8da723e98e2fc31baf8d98fdf: Status 404 returned error can't find the container with id 1f1eabb45902560a52f36b9e23786a983cb6eeb8da723e98e2fc31baf8d98fdf
Jul 08 07:02:42.012869 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:42.012834    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765c4ccff4-242ps" event={"ID":"1c9142b1-8507-43d0-92a8-851fb514b3c4","Type":"ContainerStarted","Data":"7117a6444cc43f85a3d27798fac7b5e4064772307f2df7c225b1d487ce6cb411"}
Jul 08 07:02:42.013044 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:42.012880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765c4ccff4-242ps" event={"ID":"1c9142b1-8507-43d0-92a8-851fb514b3c4","Type":"ContainerStarted","Data":"1f1eabb45902560a52f36b9e23786a983cb6eeb8da723e98e2fc31baf8d98fdf"}
Jul 08 07:02:42.033128 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:42.033062    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-765c4ccff4-242ps" podStartSLOduration=1.033047611 podStartE2EDuration="1.033047611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:02:42.031829023 +0000 UTC m=+489.631118727" watchObservedRunningTime="2026-07-08 07:02:42.033047611 +0000 UTC m=+489.632337313"
Jul 08 07:02:46.010003 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:46.009976    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j2hff"
Jul 08 07:02:51.644439 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:51.644404    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:51.644917 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:51.644455    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:51.649262 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:51.649241    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:52.056591 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:52.056520    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-765c4ccff4-242ps"
Jul 08 07:02:54.580645 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.580608    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"]
Jul 08 07:02:54.587490 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.587471    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.590223 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.590203    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 08 07:02:54.591516 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.591494    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rxp6h\""
Jul 08 07:02:54.591622 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.591498    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 08 07:02:54.593630 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.593602    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"]
Jul 08 07:02:54.703506 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.703476    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2k8\" (UniqueName: \"kubernetes.io/projected/421a4d10-350e-494a-ab78-906c728552a8-kube-api-access-9r2k8\") pod \"openshift-lws-operator-fd8ccff4c-p86rw\" (UID: \"421a4d10-350e-494a-ab78-906c728552a8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.703656 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.703515    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/421a4d10-350e-494a-ab78-906c728552a8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-p86rw\" (UID: \"421a4d10-350e-494a-ab78-906c728552a8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.804361 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.804336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2k8\" (UniqueName: \"kubernetes.io/projected/421a4d10-350e-494a-ab78-906c728552a8-kube-api-access-9r2k8\") pod \"openshift-lws-operator-fd8ccff4c-p86rw\" (UID: \"421a4d10-350e-494a-ab78-906c728552a8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.804513 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.804370    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/421a4d10-350e-494a-ab78-906c728552a8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-p86rw\" (UID: \"421a4d10-350e-494a-ab78-906c728552a8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.804708 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.804667    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/421a4d10-350e-494a-ab78-906c728552a8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-p86rw\" (UID: \"421a4d10-350e-494a-ab78-906c728552a8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.812760 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.812728    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2k8\" (UniqueName: \"kubernetes.io/projected/421a4d10-350e-494a-ab78-906c728552a8-kube-api-access-9r2k8\") pod \"openshift-lws-operator-fd8ccff4c-p86rw\" (UID: \"421a4d10-350e-494a-ab78-906c728552a8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:54.897779 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:54.897753    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"
Jul 08 07:02:55.029875 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:55.029851    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw"]
Jul 08 07:02:55.032466 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:02:55.032441    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod421a4d10_350e_494a_ab78_906c728552a8.slice/crio-5e4f7290d547be2398f63369bf12c4608076e3ad1382605f4e7f1d5ab77ccdb0 WatchSource:0}: Error finding container 5e4f7290d547be2398f63369bf12c4608076e3ad1382605f4e7f1d5ab77ccdb0: Status 404 returned error can't find the container with id 5e4f7290d547be2398f63369bf12c4608076e3ad1382605f4e7f1d5ab77ccdb0
Jul 08 07:02:55.065949 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:55.065919    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw" event={"ID":"421a4d10-350e-494a-ab78-906c728552a8","Type":"ContainerStarted","Data":"5e4f7290d547be2398f63369bf12c4608076e3ad1382605f4e7f1d5ab77ccdb0"}
Jul 08 07:02:58.081107 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:58.081065    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw" event={"ID":"421a4d10-350e-494a-ab78-906c728552a8","Type":"ContainerStarted","Data":"69ea55fa480fd419caedcc7164513cc1c5444ea809913dbaae11713c0a641533"}
Jul 08 07:02:58.099067 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:02:58.099022    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p86rw" podStartSLOduration=1.5450694280000001 podStartE2EDuration="4.099007091s" totalImagesPullingTime="2.553937663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:02:58.097586806 +0000 UTC m=+505.696876512" watchObservedRunningTime="2026-07-08 07:02:58.099007091 +0000 UTC m=+505.698296796"
Jul 08 07:03:20.420327 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.420291    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"]
Jul 08 07:03:20.422630 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.422612    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.425184 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.425163    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 08 07:03:20.425292 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.425275    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 08 07:03:20.426368 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.426351    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nh4tx\""
Jul 08 07:03:20.426443 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.426429    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 08 07:03:20.433052 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.433030    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"]
Jul 08 07:03:20.542666 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.542633    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18d572c8-8155-4737-8260-cdf19c0ed818-metrics-cert\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.542848 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.542705    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18d572c8-8155-4737-8260-cdf19c0ed818-cert\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.542848 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.542750    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18d572c8-8155-4737-8260-cdf19c0ed818-manager-config\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.542848 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.542767    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l98m\" (UniqueName: \"kubernetes.io/projected/18d572c8-8155-4737-8260-cdf19c0ed818-kube-api-access-5l98m\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.643996 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.643957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18d572c8-8155-4737-8260-cdf19c0ed818-manager-config\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.644188 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.644001    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l98m\" (UniqueName: \"kubernetes.io/projected/18d572c8-8155-4737-8260-cdf19c0ed818-kube-api-access-5l98m\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.644188 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.644065    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18d572c8-8155-4737-8260-cdf19c0ed818-metrics-cert\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.644188 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.644118    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18d572c8-8155-4737-8260-cdf19c0ed818-cert\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.644667 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.644644    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18d572c8-8155-4737-8260-cdf19c0ed818-manager-config\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.646703 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.646668    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18d572c8-8155-4737-8260-cdf19c0ed818-cert\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.646778 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.646740    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18d572c8-8155-4737-8260-cdf19c0ed818-metrics-cert\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.653341 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.653320    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l98m\" (UniqueName: \"kubernetes.io/projected/18d572c8-8155-4737-8260-cdf19c0ed818-kube-api-access-5l98m\") pod \"lws-controller-manager-58c59b5fc8-fbnwl\" (UID: \"18d572c8-8155-4737-8260-cdf19c0ed818\") " pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.732442 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.732369    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:20.875227 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:20.875203    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"]
Jul 08 07:03:20.877700 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:03:20.877658    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d572c8_8155_4737_8260_cdf19c0ed818.slice/crio-0eb81b8f4829232cf8a9b61ea43788199c1607549699d50bcf8d7b57aa8400c7 WatchSource:0}: Error finding container 0eb81b8f4829232cf8a9b61ea43788199c1607549699d50bcf8d7b57aa8400c7: Status 404 returned error can't find the container with id 0eb81b8f4829232cf8a9b61ea43788199c1607549699d50bcf8d7b57aa8400c7
Jul 08 07:03:21.177380 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:21.177334    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl" event={"ID":"18d572c8-8155-4737-8260-cdf19c0ed818","Type":"ContainerStarted","Data":"0eb81b8f4829232cf8a9b61ea43788199c1607549699d50bcf8d7b57aa8400c7"}
Jul 08 07:03:24.194048 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:24.193905    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl" event={"ID":"18d572c8-8155-4737-8260-cdf19c0ed818","Type":"ContainerStarted","Data":"9047eeb7d01d4782d46402e520a2ec24fff9664f3851f2347e84a4008a508d79"}
Jul 08 07:03:24.194502 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:24.194151    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:03:24.212836 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:24.212788    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl" podStartSLOduration=1.9331911069999999 podStartE2EDuration="4.212772409s" totalImagesPullingTime="2.279581302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:03:24.210962321 +0000 UTC m=+531.810252023" watchObservedRunningTime="2026-07-08 07:03:24.212772409 +0000 UTC m=+531.812062111"
Jul 08 07:03:35.200936 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:03:35.200904    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58c59b5fc8-fbnwl"
Jul 08 07:04:06.917148 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:06.917115    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"]
Jul 08 07:04:06.920901 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:06.920883    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:06.923594 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:06.923564    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 07:04:06.923817 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:06.923798    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4wnlk\""
Jul 08 07:04:06.924109 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:06.924096    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 07:04:06.937608 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:06.937585    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"]
Jul 08 07:04:07.052811 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.052773    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndmf6\" (UniqueName: \"kubernetes.io/projected/46ca0a88-03ba-4de1-a7e0-fdf989eccf31-kube-api-access-ndmf6\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-cvspb\" (UID: \"46ca0a88-03ba-4de1-a7e0-fdf989eccf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.053015 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.052829    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46ca0a88-03ba-4de1-a7e0-fdf989eccf31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-cvspb\" (UID: \"46ca0a88-03ba-4de1-a7e0-fdf989eccf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.154087 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.154052    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndmf6\" (UniqueName: \"kubernetes.io/projected/46ca0a88-03ba-4de1-a7e0-fdf989eccf31-kube-api-access-ndmf6\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-cvspb\" (UID: \"46ca0a88-03ba-4de1-a7e0-fdf989eccf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.154087 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.154091    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46ca0a88-03ba-4de1-a7e0-fdf989eccf31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-cvspb\" (UID: \"46ca0a88-03ba-4de1-a7e0-fdf989eccf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.154409 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.154393    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46ca0a88-03ba-4de1-a7e0-fdf989eccf31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-cvspb\" (UID: \"46ca0a88-03ba-4de1-a7e0-fdf989eccf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.164404 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.164379    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndmf6\" (UniqueName: \"kubernetes.io/projected/46ca0a88-03ba-4de1-a7e0-fdf989eccf31-kube-api-access-ndmf6\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-cvspb\" (UID: \"46ca0a88-03ba-4de1-a7e0-fdf989eccf31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.230883 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.230806    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:07.370393 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.370369    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"]
Jul 08 07:04:07.372806 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:04:07.372780    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ca0a88_03ba_4de1_a7e0_fdf989eccf31.slice/crio-600cdbb60ba95351815efa2fc6c9e0ad0656b9604a506322c1c8a5b6f48ea260 WatchSource:0}: Error finding container 600cdbb60ba95351815efa2fc6c9e0ad0656b9604a506322c1c8a5b6f48ea260: Status 404 returned error can't find the container with id 600cdbb60ba95351815efa2fc6c9e0ad0656b9604a506322c1c8a5b6f48ea260
Jul 08 07:04:07.379418 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:07.379394    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb" event={"ID":"46ca0a88-03ba-4de1-a7e0-fdf989eccf31","Type":"ContainerStarted","Data":"600cdbb60ba95351815efa2fc6c9e0ad0656b9604a506322c1c8a5b6f48ea260"}
Jul 08 07:04:18.442249 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:18.442216    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb" event={"ID":"46ca0a88-03ba-4de1-a7e0-fdf989eccf31","Type":"ContainerStarted","Data":"1e139f01efdefa0d2fd7ebeef5841ee76efaad0a5d74d4ba092879a3b724abfc"}
Jul 08 07:04:18.442607 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:18.442350    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:04:18.462324 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:18.462271    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb" podStartSLOduration=1.8615680079999999 podStartE2EDuration="12.462256917s" totalImagesPullingTime="10.600688909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:04:18.46000103 +0000 UTC m=+586.059290730" watchObservedRunningTime="2026-07-08 07:04:18.462256917 +0000 UTC m=+586.061546621"
Jul 08 07:04:29.450382 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:04:29.450353    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-cvspb"
Jul 08 07:05:04.261459 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.261387    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:04.263572 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.263553    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.266298 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.266283    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 08 07:05:04.266448 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.266432    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mps7s\""
Jul 08 07:05:04.274504 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.274485    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:04.358379 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.358349    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:04.360115 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.360091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm85z\" (UniqueName: \"kubernetes.io/projected/7b858db1-767d-4b29-86a0-6974f53b3211-kube-api-access-gm85z\") pod \"limitador-limitador-ff68ff5cb-tc75h\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.360253 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.360131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b858db1-767d-4b29-86a0-6974f53b3211-config-file\") pod \"limitador-limitador-ff68ff5cb-tc75h\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.461407 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.461368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm85z\" (UniqueName: \"kubernetes.io/projected/7b858db1-767d-4b29-86a0-6974f53b3211-kube-api-access-gm85z\") pod \"limitador-limitador-ff68ff5cb-tc75h\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.461407 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.461406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b858db1-767d-4b29-86a0-6974f53b3211-config-file\") pod \"limitador-limitador-ff68ff5cb-tc75h\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.462011 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.461991    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b858db1-767d-4b29-86a0-6974f53b3211-config-file\") pod \"limitador-limitador-ff68ff5cb-tc75h\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.469912 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.469891    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm85z\" (UniqueName: \"kubernetes.io/projected/7b858db1-767d-4b29-86a0-6974f53b3211-kube-api-access-gm85z\") pod \"limitador-limitador-ff68ff5cb-tc75h\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.573953 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.573879    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:04.713007 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:04.712980    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:04.715237 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:05:04.715215    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b858db1_767d_4b29_86a0_6974f53b3211.slice/crio-1b3b07c386bfc564a7759c05b8635c9ab46adfc0b4014212d1f18abbe0191582 WatchSource:0}: Error finding container 1b3b07c386bfc564a7759c05b8635c9ab46adfc0b4014212d1f18abbe0191582: Status 404 returned error can't find the container with id 1b3b07c386bfc564a7759c05b8635c9ab46adfc0b4014212d1f18abbe0191582
Jul 08 07:05:05.195844 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.195811    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8df697c9f-bgtf2"]
Jul 08 07:05:05.198743 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.198718    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:05.201665 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.201647    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cxt5w\""
Jul 08 07:05:05.206795 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.206764    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8df697c9f-bgtf2"]
Jul 08 07:05:05.267924 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.267896    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxcdx\" (UniqueName: \"kubernetes.io/projected/21431a6e-f525-4a49-99e3-47b0935e8d48-kube-api-access-gxcdx\") pod \"authorino-8df697c9f-bgtf2\" (UID: \"21431a6e-f525-4a49-99e3-47b0935e8d48\") " pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:05.368559 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.368524    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxcdx\" (UniqueName: \"kubernetes.io/projected/21431a6e-f525-4a49-99e3-47b0935e8d48-kube-api-access-gxcdx\") pod \"authorino-8df697c9f-bgtf2\" (UID: \"21431a6e-f525-4a49-99e3-47b0935e8d48\") " pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:05.377493 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.377463    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxcdx\" (UniqueName: \"kubernetes.io/projected/21431a6e-f525-4a49-99e3-47b0935e8d48-kube-api-access-gxcdx\") pod \"authorino-8df697c9f-bgtf2\" (UID: \"21431a6e-f525-4a49-99e3-47b0935e8d48\") " pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:05.509203 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.509115    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:05.640199 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.640160    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8df697c9f-bgtf2"]
Jul 08 07:05:05.643806 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:05:05.643777    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21431a6e_f525_4a49_99e3_47b0935e8d48.slice/crio-0b9259f474c144ccc415e96cd3b76368dd1c55b41587fbcb44182ceaba928721 WatchSource:0}: Error finding container 0b9259f474c144ccc415e96cd3b76368dd1c55b41587fbcb44182ceaba928721: Status 404 returned error can't find the container with id 0b9259f474c144ccc415e96cd3b76368dd1c55b41587fbcb44182ceaba928721
Jul 08 07:05:05.670975 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.670948    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-bgtf2" event={"ID":"21431a6e-f525-4a49-99e3-47b0935e8d48","Type":"ContainerStarted","Data":"0b9259f474c144ccc415e96cd3b76368dd1c55b41587fbcb44182ceaba928721"}
Jul 08 07:05:05.672088 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:05.672065    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h" event={"ID":"7b858db1-767d-4b29-86a0-6974f53b3211","Type":"ContainerStarted","Data":"1b3b07c386bfc564a7759c05b8635c9ab46adfc0b4014212d1f18abbe0191582"}
Jul 08 07:05:10.701495 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:10.701449    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-bgtf2" event={"ID":"21431a6e-f525-4a49-99e3-47b0935e8d48","Type":"ContainerStarted","Data":"3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62"}
Jul 08 07:05:10.703004 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:10.702981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h" event={"ID":"7b858db1-767d-4b29-86a0-6974f53b3211","Type":"ContainerStarted","Data":"e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c"}
Jul 08 07:05:10.703122 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:10.703053    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:10.732379 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:10.732320    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8df697c9f-bgtf2" podStartSLOduration=1.177116546 podStartE2EDuration="5.732298564s" totalImagesPullingTime="4.555182018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:05:10.716483568 +0000 UTC m=+638.315773273" watchObservedRunningTime="2026-07-08 07:05:10.732298564 +0000 UTC m=+638.331588282"
Jul 08 07:05:10.733479 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:10.733447    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h" podStartSLOduration=1.200464265 podStartE2EDuration="6.733435053s" totalImagesPullingTime="5.532970788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:05:10.731517091 +0000 UTC m=+638.330806806" watchObservedRunningTime="2026-07-08 07:05:10.733435053 +0000 UTC m=+638.332724756"
Jul 08 07:05:20.818833 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:20.818797    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:20.821413 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:20.819039    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h" podUID="7b858db1-767d-4b29-86a0-6974f53b3211" containerName="limitador" containerID="cri-o://e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c" gracePeriod=30
Jul 08 07:05:20.821413 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:20.820913    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:21.374128 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.374097    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:21.518870 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.518774    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b858db1-767d-4b29-86a0-6974f53b3211-config-file\") pod \"7b858db1-767d-4b29-86a0-6974f53b3211\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") "
Jul 08 07:05:21.519017 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.518896    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm85z\" (UniqueName: \"kubernetes.io/projected/7b858db1-767d-4b29-86a0-6974f53b3211-kube-api-access-gm85z\") pod \"7b858db1-767d-4b29-86a0-6974f53b3211\" (UID: \"7b858db1-767d-4b29-86a0-6974f53b3211\") "
Jul 08 07:05:21.519148 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.519122    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b858db1-767d-4b29-86a0-6974f53b3211-config-file" (OuterVolumeSpecName: "config-file") pod "7b858db1-767d-4b29-86a0-6974f53b3211" (UID: "7b858db1-767d-4b29-86a0-6974f53b3211"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 07:05:21.521180 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.521157    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b858db1-767d-4b29-86a0-6974f53b3211-kube-api-access-gm85z" (OuterVolumeSpecName: "kube-api-access-gm85z") pod "7b858db1-767d-4b29-86a0-6974f53b3211" (UID: "7b858db1-767d-4b29-86a0-6974f53b3211"). InnerVolumeSpecName "kube-api-access-gm85z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:05:21.620522 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.620479    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm85z\" (UniqueName: \"kubernetes.io/projected/7b858db1-767d-4b29-86a0-6974f53b3211-kube-api-access-gm85z\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:05:21.620522 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.620514    2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7b858db1-767d-4b29-86a0-6974f53b3211-config-file\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:05:21.761364 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.761331    2571 generic.go:356] "Generic (PLEG): container finished" podID="7b858db1-767d-4b29-86a0-6974f53b3211" containerID="e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c" exitCode=0
Jul 08 07:05:21.761526 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.761372    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"
Jul 08 07:05:21.761526 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.761409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h" event={"ID":"7b858db1-767d-4b29-86a0-6974f53b3211","Type":"ContainerDied","Data":"e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c"}
Jul 08 07:05:21.761526 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.761455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-tc75h" event={"ID":"7b858db1-767d-4b29-86a0-6974f53b3211","Type":"ContainerDied","Data":"1b3b07c386bfc564a7759c05b8635c9ab46adfc0b4014212d1f18abbe0191582"}
Jul 08 07:05:21.761526 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.761477    2571 scope.go:117] "RemoveContainer" containerID="e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c"
Jul 08 07:05:21.770723 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.770706    2571 scope.go:117] "RemoveContainer" containerID="e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c"
Jul 08 07:05:21.771026 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:05:21.771009    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c\": container with ID starting with e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c not found: ID does not exist" containerID="e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c"
Jul 08 07:05:21.771071 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.771035    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c"} err="failed to get container status \"e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c\": rpc error: code = NotFound desc = could not find container \"e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c\": container with ID starting with e1e0be8d7358ec21e8b4ef35b25943e758b91c350fcd4fa724c1e34774ba5c3c not found: ID does not exist"
Jul 08 07:05:21.783327 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.783280    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:21.785308 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:21.785290    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-tc75h"]
Jul 08 07:05:22.913861 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:22.913831    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b858db1-767d-4b29-86a0-6974f53b3211" path="/var/lib/kubelet/pods/7b858db1-767d-4b29-86a0-6974f53b3211/volumes"
Jul 08 07:05:32.122270 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.122236    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8df697c9f-bgtf2"]
Jul 08 07:05:32.122745 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.122448    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-8df697c9f-bgtf2" podUID="21431a6e-f525-4a49-99e3-47b0935e8d48" containerName="authorino" containerID="cri-o://3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62" gracePeriod=30
Jul 08 07:05:32.370110 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.370087    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:32.521441 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.521365    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxcdx\" (UniqueName: \"kubernetes.io/projected/21431a6e-f525-4a49-99e3-47b0935e8d48-kube-api-access-gxcdx\") pod \"21431a6e-f525-4a49-99e3-47b0935e8d48\" (UID: \"21431a6e-f525-4a49-99e3-47b0935e8d48\") "
Jul 08 07:05:32.523595 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.523558    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21431a6e-f525-4a49-99e3-47b0935e8d48-kube-api-access-gxcdx" (OuterVolumeSpecName: "kube-api-access-gxcdx") pod "21431a6e-f525-4a49-99e3-47b0935e8d48" (UID: "21431a6e-f525-4a49-99e3-47b0935e8d48"). InnerVolumeSpecName "kube-api-access-gxcdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:05:32.622298 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.622262    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxcdx\" (UniqueName: \"kubernetes.io/projected/21431a6e-f525-4a49-99e3-47b0935e8d48-kube-api-access-gxcdx\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:05:32.813046 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.812971    2571 generic.go:356] "Generic (PLEG): container finished" podID="21431a6e-f525-4a49-99e3-47b0935e8d48" containerID="3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62" exitCode=0
Jul 08 07:05:32.813046 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.813016    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-bgtf2"
Jul 08 07:05:32.813214 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.813062    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-bgtf2" event={"ID":"21431a6e-f525-4a49-99e3-47b0935e8d48","Type":"ContainerDied","Data":"3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62"}
Jul 08 07:05:32.813214 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.813103    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-bgtf2" event={"ID":"21431a6e-f525-4a49-99e3-47b0935e8d48","Type":"ContainerDied","Data":"0b9259f474c144ccc415e96cd3b76368dd1c55b41587fbcb44182ceaba928721"}
Jul 08 07:05:32.813214 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.813122    2571 scope.go:117] "RemoveContainer" containerID="3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62"
Jul 08 07:05:32.822985 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.822964    2571 scope.go:117] "RemoveContainer" containerID="3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62"
Jul 08 07:05:32.823261 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:05:32.823241    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62\": container with ID starting with 3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62 not found: ID does not exist" containerID="3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62"
Jul 08 07:05:32.823335 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.823268    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62"} err="failed to get container status \"3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62\": rpc error: code = NotFound desc = could not find container \"3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62\": container with ID starting with 3b7ae531f857466afd0db625009ceea1eb8ec15bf87d962526e0e27c17172d62 not found: ID does not exist"
Jul 08 07:05:32.833786 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.833762    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8df697c9f-bgtf2"]
Jul 08 07:05:32.837308 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.837288    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8df697c9f-bgtf2"]
Jul 08 07:05:32.915223 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:05:32.915191    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21431a6e-f525-4a49-99e3-47b0935e8d48" path="/var/lib/kubelet/pods/21431a6e-f525-4a49-99e3-47b0935e8d48/volumes"
Jul 08 07:06:10.146271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146240    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-4tbxf"]
Jul 08 07:06:10.146761 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146578    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21431a6e-f525-4a49-99e3-47b0935e8d48" containerName="authorino"
Jul 08 07:06:10.146761 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146591    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="21431a6e-f525-4a49-99e3-47b0935e8d48" containerName="authorino"
Jul 08 07:06:10.146761 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146605    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b858db1-767d-4b29-86a0-6974f53b3211" containerName="limitador"
Jul 08 07:06:10.146761 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146611    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b858db1-767d-4b29-86a0-6974f53b3211" containerName="limitador"
Jul 08 07:06:10.146761 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146677    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b858db1-767d-4b29-86a0-6974f53b3211" containerName="limitador"
Jul 08 07:06:10.146761 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.146705    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="21431a6e-f525-4a49-99e3-47b0935e8d48" containerName="authorino"
Jul 08 07:06:10.148517 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.148501    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.151206 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.151156    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 08 07:06:10.151335 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.151232    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-t86cf\""
Jul 08 07:06:10.151335 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.151266    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 08 07:06:10.152116 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.152098    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 08 07:06:10.158332 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.158311    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-4tbxf"]
Jul 08 07:06:10.163958 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.163933    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-9rsxr"]
Jul 08 07:06:10.166131 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.166114    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.168562 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.168539    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 08 07:06:10.168678 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.168597    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-c6zdt\""
Jul 08 07:06:10.176264 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.176241    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-9rsxr"]
Jul 08 07:06:10.266381 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.266346    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4a13e5c-3a04-43dd-95d4-bbe56c03898f-cert\") pod \"odh-model-controller-84b54b7f97-9rsxr\" (UID: \"c4a13e5c-3a04-43dd-95d4-bbe56c03898f\") " pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.266381 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.266380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsd62\" (UniqueName: \"kubernetes.io/projected/c4a13e5c-3a04-43dd-95d4-bbe56c03898f-kube-api-access-hsd62\") pod \"odh-model-controller-84b54b7f97-9rsxr\" (UID: \"c4a13e5c-3a04-43dd-95d4-bbe56c03898f\") " pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.266594 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.266413    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a79acbc-c478-430d-9726-96d07eb6a90b-tls-certs\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.266594 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.266467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2zc7\" (UniqueName: \"kubernetes.io/projected/2a79acbc-c478-430d-9726-96d07eb6a90b-kube-api-access-r2zc7\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.367577 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.367541    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4a13e5c-3a04-43dd-95d4-bbe56c03898f-cert\") pod \"odh-model-controller-84b54b7f97-9rsxr\" (UID: \"c4a13e5c-3a04-43dd-95d4-bbe56c03898f\") " pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.367577 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.367577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsd62\" (UniqueName: \"kubernetes.io/projected/c4a13e5c-3a04-43dd-95d4-bbe56c03898f-kube-api-access-hsd62\") pod \"odh-model-controller-84b54b7f97-9rsxr\" (UID: \"c4a13e5c-3a04-43dd-95d4-bbe56c03898f\") " pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.367808 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.367632    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a79acbc-c478-430d-9726-96d07eb6a90b-tls-certs\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.367808 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.367677    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2zc7\" (UniqueName: \"kubernetes.io/projected/2a79acbc-c478-430d-9726-96d07eb6a90b-kube-api-access-r2zc7\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.367808 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:06:10.367782    2571 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 08 07:06:10.367909 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:06:10.367867    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a79acbc-c478-430d-9726-96d07eb6a90b-tls-certs podName:2a79acbc-c478-430d-9726-96d07eb6a90b nodeName:}" failed. No retries permitted until 2026-07-08 07:06:10.867845289 +0000 UTC m=+698.467134970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2a79acbc-c478-430d-9726-96d07eb6a90b-tls-certs") pod "model-serving-api-fd65d7d6-4tbxf" (UID: "2a79acbc-c478-430d-9726-96d07eb6a90b") : secret "model-serving-api-tls" not found
Jul 08 07:06:10.370250 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.370224    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4a13e5c-3a04-43dd-95d4-bbe56c03898f-cert\") pod \"odh-model-controller-84b54b7f97-9rsxr\" (UID: \"c4a13e5c-3a04-43dd-95d4-bbe56c03898f\") " pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.378527 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.378504    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2zc7\" (UniqueName: \"kubernetes.io/projected/2a79acbc-c478-430d-9726-96d07eb6a90b-kube-api-access-r2zc7\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.378772 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.378753    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsd62\" (UniqueName: \"kubernetes.io/projected/c4a13e5c-3a04-43dd-95d4-bbe56c03898f-kube-api-access-hsd62\") pod \"odh-model-controller-84b54b7f97-9rsxr\" (UID: \"c4a13e5c-3a04-43dd-95d4-bbe56c03898f\") " pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.476357 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.476263    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:10.607968 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.607884    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-9rsxr"]
Jul 08 07:06:10.610191 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:06:10.610165    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a13e5c_3a04_43dd_95d4_bbe56c03898f.slice/crio-8d877d1533d9dbb2a76e056f0fa2d5bf5cfdeafdd7734e3b8e1f53892e118f62 WatchSource:0}: Error finding container 8d877d1533d9dbb2a76e056f0fa2d5bf5cfdeafdd7734e3b8e1f53892e118f62: Status 404 returned error can't find the container with id 8d877d1533d9dbb2a76e056f0fa2d5bf5cfdeafdd7734e3b8e1f53892e118f62
Jul 08 07:06:10.872780 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.872744    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a79acbc-c478-430d-9726-96d07eb6a90b-tls-certs\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.875340 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.875318    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a79acbc-c478-430d-9726-96d07eb6a90b-tls-certs\") pod \"model-serving-api-fd65d7d6-4tbxf\" (UID: \"2a79acbc-c478-430d-9726-96d07eb6a90b\") " pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:10.981782 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:10.981748    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-9rsxr" event={"ID":"c4a13e5c-3a04-43dd-95d4-bbe56c03898f","Type":"ContainerStarted","Data":"8d877d1533d9dbb2a76e056f0fa2d5bf5cfdeafdd7734e3b8e1f53892e118f62"}
Jul 08 07:06:11.059749 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:11.059719    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:11.189886 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:11.189775    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-4tbxf"]
Jul 08 07:06:11.191959 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:06:11.191931    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a79acbc_c478_430d_9726_96d07eb6a90b.slice/crio-bb1c43205d796f46ec72c7fbf9a15b2cb4423343bcca7e162fc061ab03eb6e80 WatchSource:0}: Error finding container bb1c43205d796f46ec72c7fbf9a15b2cb4423343bcca7e162fc061ab03eb6e80: Status 404 returned error can't find the container with id bb1c43205d796f46ec72c7fbf9a15b2cb4423343bcca7e162fc061ab03eb6e80
Jul 08 07:06:11.990299 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:11.990217    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-4tbxf" event={"ID":"2a79acbc-c478-430d-9726-96d07eb6a90b","Type":"ContainerStarted","Data":"bb1c43205d796f46ec72c7fbf9a15b2cb4423343bcca7e162fc061ab03eb6e80"}
Jul 08 07:06:12.997599 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:12.997525    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-4tbxf" event={"ID":"2a79acbc-c478-430d-9726-96d07eb6a90b","Type":"ContainerStarted","Data":"2360d31c675b339e9a92eaa9a969c3a73cc77bf9736fbb55f74c047fa1585915"}
Jul 08 07:06:12.997599 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:12.997595    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:13.015777 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:13.015716    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-4tbxf" podStartSLOduration=1.59348735 podStartE2EDuration="3.01569897s" totalImagesPullingTime="1.42221162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:06:13.013601624 +0000 UTC m=+700.612891326" watchObservedRunningTime="2026-07-08 07:06:13.01569897 +0000 UTC m=+700.614988664"
Jul 08 07:06:15.008968 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:15.008936    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-9rsxr" event={"ID":"c4a13e5c-3a04-43dd-95d4-bbe56c03898f","Type":"ContainerStarted","Data":"c1e3fe7dabd082d18ff8ccb87065e08422e1b638f06ef3c408e0706a0ad2528b"}
Jul 08 07:06:15.009359 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:15.009105    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:15.028152 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:15.028094    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-9rsxr" podStartSLOduration=1.6005606860000001 podStartE2EDuration="5.028076363s" totalImagesPullingTime="3.427515677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:06:15.027939337 +0000 UTC m=+702.627229040" watchObservedRunningTime="2026-07-08 07:06:15.028076363 +0000 UTC m=+702.627366069"
Jul 08 07:06:24.007473 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:24.007394    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-4tbxf"
Jul 08 07:06:26.016137 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.016109    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-9rsxr"
Jul 08 07:06:26.808745 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.808708    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-7t9ph"]
Jul 08 07:06:26.812064 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.812047    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-7t9ph"
Jul 08 07:06:26.814515 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.814490    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c24181de-68b3-464d-a375-783658f553eb-model-cache\") pod \"s3-init-7t9ph\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") " pod="kserve/s3-init-7t9ph"
Jul 08 07:06:26.814618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.814525    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-7pcq6\""
Jul 08 07:06:26.814618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.814551    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 08 07:06:26.814618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.814558    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr6g4\" (UniqueName: \"kubernetes.io/projected/c24181de-68b3-464d-a375-783658f553eb-kube-api-access-qr6g4\") pod \"s3-init-7t9ph\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") " pod="kserve/s3-init-7t9ph"
Jul 08 07:06:26.819255 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.819223    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-7t9ph"]
Jul 08 07:06:26.915531 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.915501    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr6g4\" (UniqueName: \"kubernetes.io/projected/c24181de-68b3-464d-a375-783658f553eb-kube-api-access-qr6g4\") pod \"s3-init-7t9ph\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") " pod="kserve/s3-init-7t9ph"
Jul 08 07:06:26.915679 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.915575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c24181de-68b3-464d-a375-783658f553eb-model-cache\") pod \"s3-init-7t9ph\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") " pod="kserve/s3-init-7t9ph"
Jul 08 07:06:26.915885 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.915870    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c24181de-68b3-464d-a375-783658f553eb-model-cache\") pod \"s3-init-7t9ph\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") " pod="kserve/s3-init-7t9ph"
Jul 08 07:06:26.923655 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:26.923625    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr6g4\" (UniqueName: \"kubernetes.io/projected/c24181de-68b3-464d-a375-783658f553eb-kube-api-access-qr6g4\") pod \"s3-init-7t9ph\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") " pod="kserve/s3-init-7t9ph"
Jul 08 07:06:27.122741 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:27.122705    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-7t9ph"
Jul 08 07:06:27.256036 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:27.256007    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-7t9ph"]
Jul 08 07:06:27.258414 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:06:27.258384    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc24181de_68b3_464d_a375_783658f553eb.slice/crio-c04cbc3e1efb1719ff1685fb46bc3d69ecf0a2565c71d46e56a95a18abd0a551 WatchSource:0}: Error finding container c04cbc3e1efb1719ff1685fb46bc3d69ecf0a2565c71d46e56a95a18abd0a551: Status 404 returned error can't find the container with id c04cbc3e1efb1719ff1685fb46bc3d69ecf0a2565c71d46e56a95a18abd0a551
Jul 08 07:06:28.071706 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:28.071654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-7t9ph" event={"ID":"c24181de-68b3-464d-a375-783658f553eb","Type":"ContainerStarted","Data":"c04cbc3e1efb1719ff1685fb46bc3d69ecf0a2565c71d46e56a95a18abd0a551"}
Jul 08 07:06:31.089900 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:31.089810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-7t9ph" event={"ID":"c24181de-68b3-464d-a375-783658f553eb","Type":"ContainerStarted","Data":"6f3a722666ca789fcb12c2993644f02ba5486ff5ec290df763263d01dca11544"}
Jul 08 07:06:33.100296 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:33.100261    2571 generic.go:356] "Generic (PLEG): container finished" podID="c24181de-68b3-464d-a375-783658f553eb" containerID="6f3a722666ca789fcb12c2993644f02ba5486ff5ec290df763263d01dca11544" exitCode=0
Jul 08 07:06:33.100722 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:33.100324    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-7t9ph" event={"ID":"c24181de-68b3-464d-a375-783658f553eb","Type":"ContainerDied","Data":"6f3a722666ca789fcb12c2993644f02ba5486ff5ec290df763263d01dca11544"}
Jul 08 07:06:37.654906 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:37.654883    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 08 07:06:38.130153 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:38.130120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-7t9ph" event={"ID":"c24181de-68b3-464d-a375-783658f553eb","Type":"ContainerStarted","Data":"f3234d5e07ba5726a1b7cf9b8bfdd0817a681d186d23f28ad3f6f2a27ced6716"}
Jul 08 07:06:38.146892 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:06:38.146839    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-7t9ph" podStartSLOduration=2.813036454 podStartE2EDuration="12.146823422s" totalImagesPullingTime="7.893070265s" totalInitContainerRuntime="1.440716703s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:06:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:06:38.145780689 +0000 UTC m=+725.745070393" watchObservedRunningTime="2026-07-08 07:06:38.146823422 +0000 UTC m=+725.746113124"
Jul 08 07:07:43.435995 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:43.435956    2571 generic.go:356] "Generic (PLEG): container finished" podID="c24181de-68b3-464d-a375-783658f553eb" containerID="f3234d5e07ba5726a1b7cf9b8bfdd0817a681d186d23f28ad3f6f2a27ced6716" exitCode=0
Jul 08 07:07:43.436418 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:43.436029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-7t9ph" event={"ID":"c24181de-68b3-464d-a375-783658f553eb","Type":"ContainerDied","Data":"f3234d5e07ba5726a1b7cf9b8bfdd0817a681d186d23f28ad3f6f2a27ced6716"}
Jul 08 07:07:44.576202 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.576173    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-7t9ph"
Jul 08 07:07:44.707246 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.707157    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr6g4\" (UniqueName: \"kubernetes.io/projected/c24181de-68b3-464d-a375-783658f553eb-kube-api-access-qr6g4\") pod \"c24181de-68b3-464d-a375-783658f553eb\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") "
Jul 08 07:07:44.707246 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.707223    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c24181de-68b3-464d-a375-783658f553eb-model-cache\") pod \"c24181de-68b3-464d-a375-783658f553eb\" (UID: \"c24181de-68b3-464d-a375-783658f553eb\") "
Jul 08 07:07:44.709634 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.709598    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c24181de-68b3-464d-a375-783658f553eb-kube-api-access-qr6g4" (OuterVolumeSpecName: "kube-api-access-qr6g4") pod "c24181de-68b3-464d-a375-783658f553eb" (UID: "c24181de-68b3-464d-a375-783658f553eb"). InnerVolumeSpecName "kube-api-access-qr6g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:07:44.725823 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.725788    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c24181de-68b3-464d-a375-783658f553eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "c24181de-68b3-464d-a375-783658f553eb" (UID: "c24181de-68b3-464d-a375-783658f553eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:07:44.807848 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.807803    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr6g4\" (UniqueName: \"kubernetes.io/projected/c24181de-68b3-464d-a375-783658f553eb-kube-api-access-qr6g4\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:07:44.807848 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:44.807840    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c24181de-68b3-464d-a375-783658f553eb-model-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:07:45.446947 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:45.446915    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-7t9ph" event={"ID":"c24181de-68b3-464d-a375-783658f553eb","Type":"ContainerDied","Data":"c04cbc3e1efb1719ff1685fb46bc3d69ecf0a2565c71d46e56a95a18abd0a551"}
Jul 08 07:07:45.446947 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:45.446951    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c04cbc3e1efb1719ff1685fb46bc3d69ecf0a2565c71d46e56a95a18abd0a551"
Jul 08 07:07:45.447156 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:45.446952    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-7t9ph"
Jul 08 07:07:58.231907 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.231823    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"]
Jul 08 07:07:58.232336 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.232177    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c24181de-68b3-464d-a375-783658f553eb" containerName="download-hf-model"
Jul 08 07:07:58.232336 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.232189    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c24181de-68b3-464d-a375-783658f553eb" containerName="download-hf-model"
Jul 08 07:07:58.232336 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.232199    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c24181de-68b3-464d-a375-783658f553eb" containerName="s3-init"
Jul 08 07:07:58.232336 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.232205    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c24181de-68b3-464d-a375-783658f553eb" containerName="s3-init"
Jul 08 07:07:58.232336 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.232273    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c24181de-68b3-464d-a375-783658f553eb" containerName="s3-init"
Jul 08 07:07:58.236897 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.236770    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.241112 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.241090    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 07:07:58.241112 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.241100    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 07:07:58.241718 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.241702    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
Jul 08 07:07:58.245241 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.245219    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-ctvf2\""
Jul 08 07:07:58.254746 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.254720    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"]
Jul 08 07:07:58.328187 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328154    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsf25\" (UniqueName: \"kubernetes.io/projected/f353d789-6bd9-4653-a007-ac3dab8cbbe6-kube-api-access-bsf25\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328357 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328202    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328357 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328357 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328273    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328480 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328365    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328480 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328389    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328480 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328417    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328480 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.328671 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.328550    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.429511 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429477    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.429511 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429562    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429580    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429610    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429757    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsf25\" (UniqueName: \"kubernetes.io/projected/f353d789-6bd9-4653-a007-ac3dab8cbbe6-kube-api-access-bsf25\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.429802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.430000    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.430023    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430162 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.430081    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430733 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.430272    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.430733 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.430626    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.432294 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.432270    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.432409 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.432392    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.439341 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.439317    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f353d789-6bd9-4653-a007-ac3dab8cbbe6-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.439432 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.439321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsf25\" (UniqueName: \"kubernetes.io/projected/f353d789-6bd9-4653-a007-ac3dab8cbbe6-kube-api-access-bsf25\") pod \"router-gateway-1-openshift-default-799f46c59b-kwjm2\" (UID: \"f353d789-6bd9-4653-a007-ac3dab8cbbe6\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.549578 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.549495    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:07:58.706246 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.706156    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"]
Jul 08 07:07:58.708875 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:07:58.708843    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf353d789_6bd9_4653_a007_ac3dab8cbbe6.slice/crio-b558f3096d7f067981c566d3306b09747bdd1cb01656d6b68e3055a13ff96a88 WatchSource:0}: Error finding container b558f3096d7f067981c566d3306b09747bdd1cb01656d6b68e3055a13ff96a88: Status 404 returned error can't find the container with id b558f3096d7f067981c566d3306b09747bdd1cb01656d6b68e3055a13ff96a88
Jul 08 07:07:58.711493 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:58.711465    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 07:07:59.515264 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:07:59.515223    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerStarted","Data":"b558f3096d7f067981c566d3306b09747bdd1cb01656d6b68e3055a13ff96a88"}
Jul 08 07:08:01.445944 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:01.445907    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:01.446209 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:01.445983    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:01.446209 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:01.446009    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:01.527012 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:01.526977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerStarted","Data":"097d4f6618c1fa410137b011648dbccf8bf4580fa88eecf18599a535ec157b9a"}
Jul 08 07:08:01.549742 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:01.549652    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:01.551893 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:01.551835    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podStartSLOduration=0.817816338 podStartE2EDuration="3.551816629s" totalImagesPullingTime="2.734000291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:08:01.549551235 +0000 UTC m=+809.148840940" watchObservedRunningTime="2026-07-08 07:08:01.551816629 +0000 UTC m=+809.151106332"
Jul 08 07:08:02.555500 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:02.555418    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:03.536578 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:03.536546    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:03.537430 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:03.537411    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:12.471735 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.471697    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"]
Jul 08 07:08:12.475490 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.475454    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.478245 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.478225    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
Jul 08 07:08:12.478382 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.478246    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 07:08:12.478458 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.478276    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-psjvw\""
Jul 08 07:08:12.487037 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.486981    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"]
Jul 08 07:08:12.567325 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-home\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.567495 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567396    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tls-certs\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.567495 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567427    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-model-cache\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.567495 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.567495 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxbdj\" (UniqueName: \"kubernetes.io/projected/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kube-api-access-rxbdj\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.567627 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-dshm\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.567627 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.567577    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.668797 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.668759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.668797 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.668804    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-home\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.668850    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tls-certs\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.668872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-model-cache\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.668888    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.669064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxbdj\" (UniqueName: \"kubernetes.io/projected/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kube-api-access-rxbdj\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.669139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-dshm\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.669193    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669271 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.669244    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-model-cache\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669528 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.669440    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-home\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.669610 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.669590    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.671870 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.671844    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-dshm\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.672155 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.672132    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tls-certs\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.681240 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.681219    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxbdj\" (UniqueName: \"kubernetes.io/projected/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kube-api-access-rxbdj\") pod \"precise-prefix-cache-test-kserve-6484944dcc-s5znz\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.788803 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.788712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:12.981607 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:12.981573    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"]
Jul 08 07:08:12.985453 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:08:12.985420    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0766b965_2796_4e86_9af3_f2fe8ba2b8ee.slice/crio-c15a961d809a4151983647a6d5a182073ebd289cb21a8e1e1bb0464d3a2dca0c WatchSource:0}: Error finding container c15a961d809a4151983647a6d5a182073ebd289cb21a8e1e1bb0464d3a2dca0c: Status 404 returned error can't find the container with id c15a961d809a4151983647a6d5a182073ebd289cb21a8e1e1bb0464d3a2dca0c
Jul 08 07:08:13.591043 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.590586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" event={"ID":"0766b965-2796-4e86-9af3-f2fe8ba2b8ee","Type":"ContainerStarted","Data":"35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4"}
Jul 08 07:08:13.591043 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.590637    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" event={"ID":"0766b965-2796-4e86-9af3-f2fe8ba2b8ee","Type":"ContainerStarted","Data":"c15a961d809a4151983647a6d5a182073ebd289cb21a8e1e1bb0464d3a2dca0c"}
Jul 08 07:08:13.602369 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.602336    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"]
Jul 08 07:08:13.606697 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.606658    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.609877 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.609857    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-22hhh\""
Jul 08 07:08:13.621931 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.621908    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"]
Jul 08 07:08:13.679898 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.679867    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.680074 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.679976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.680074 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.680001    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gztw\" (UniqueName: \"kubernetes.io/projected/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kube-api-access-2gztw\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.680074 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.680040    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.680239 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.680085    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.680239 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.680132    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781160 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781127    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781324 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781212    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781324 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781236    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gztw\" (UniqueName: \"kubernetes.io/projected/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kube-api-access-2gztw\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781324 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781324 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781519 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781578 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781555    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781637 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781596    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781725 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781662    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.781783 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.781761    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.784014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.783993    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.792067 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.792039    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gztw\" (UniqueName: \"kubernetes.io/projected/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kube-api-access-2gztw\") pod \"precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:13.916773 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:13.916741    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:14.102729 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:14.102676    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"]
Jul 08 07:08:14.103570 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:08:14.103545    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919f405b_87cd_4f78_9b92_a6a400dfbaa9.slice/crio-e55167cb49286718a1114d1e29474ad8ea2075b4584b009c50df72f583c7604a WatchSource:0}: Error finding container e55167cb49286718a1114d1e29474ad8ea2075b4584b009c50df72f583c7604a: Status 404 returned error can't find the container with id e55167cb49286718a1114d1e29474ad8ea2075b4584b009c50df72f583c7604a
Jul 08 07:08:14.600267 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:14.600228    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerStarted","Data":"d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d"}
Jul 08 07:08:14.600789 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:14.600290    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerStarted","Data":"e55167cb49286718a1114d1e29474ad8ea2075b4584b009c50df72f583c7604a"}
Jul 08 07:08:15.606245 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:15.606211    2571 generic.go:356] "Generic (PLEG): container finished" podID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerID="d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d" exitCode=0
Jul 08 07:08:15.606643 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:15.606312    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerDied","Data":"d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d"}
Jul 08 07:08:18.538171 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.538130    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:08:18.628546 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.628510    2571 generic.go:356] "Generic (PLEG): container finished" podID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerID="097d4f6618c1fa410137b011648dbccf8bf4580fa88eecf18599a535ec157b9a" exitCode=0
Jul 08 07:08:18.628756 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.628586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerDied","Data":"097d4f6618c1fa410137b011648dbccf8bf4580fa88eecf18599a535ec157b9a"}
Jul 08 07:08:18.629076 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.629053    2571 scope.go:117] "RemoveContainer" containerID="097d4f6618c1fa410137b011648dbccf8bf4580fa88eecf18599a535ec157b9a"
Jul 08 07:08:18.630429 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.630393    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:18.630534 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.630487    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:18.630599 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.630557    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:18.632277 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.632250    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerStarted","Data":"faac7d276ea7ed9f6a807574eaecda9195f44d9ca2e748ab7afff6a5f6eac11f"}
Jul 08 07:08:18.634593 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.634530    2571 generic.go:356] "Generic (PLEG): container finished" podID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerID="35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4" exitCode=0
Jul 08 07:08:18.634707 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:18.634676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" event={"ID":"0766b965-2796-4e86-9af3-f2fe8ba2b8ee","Type":"ContainerDied","Data":"35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4"}
Jul 08 07:08:19.644153 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:19.644115    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerStarted","Data":"24ef4dd0f6c83b99be1ef5114a33d1d4428355529c38107fe7b6fdbbd39a662e"}
Jul 08 07:08:20.550014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:20.549973    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:20.556026 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:20.555998    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:20.653708 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:20.652147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:20.653708 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:20.653108    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:21.659866 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:21.659825    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" event={"ID":"0766b965-2796-4e86-9af3-f2fe8ba2b8ee","Type":"ContainerStarted","Data":"335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e"}
Jul 08 07:08:21.682634 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:21.682587    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" podStartSLOduration=2.658520992 podStartE2EDuration="9.682570521s" totalImagesPullingTime="2.451567881s" totalInitContainerRuntime="4.572481648s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:08:21.680249594 +0000 UTC m=+829.279539297" watchObservedRunningTime="2026-07-08 07:08:21.682570521 +0000 UTC m=+829.281860223"
Jul 08 07:08:21.789781 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:21.789746    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:22.789135 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:22.789090    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:22.808170 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:22.808143    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:23.686780 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:23.686750    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:08:28.705235 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:28.705156    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj_919f405b-87cd-4f78-9b92-a6a400dfbaa9/main/0.log"
Jul 08 07:08:28.705748 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:28.705595    2571 generic.go:356] "Generic (PLEG): container finished" podID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerID="faac7d276ea7ed9f6a807574eaecda9195f44d9ca2e748ab7afff6a5f6eac11f" exitCode=1
Jul 08 07:08:28.705748 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:28.705726    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerDied","Data":"faac7d276ea7ed9f6a807574eaecda9195f44d9ca2e748ab7afff6a5f6eac11f"}
Jul 08 07:08:36.692904 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:36.692836    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 07:08:50.653958 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:50.653917    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:08:51.841236 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:51.841199    2571 generic.go:356] "Generic (PLEG): container finished" podID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerID="24ef4dd0f6c83b99be1ef5114a33d1d4428355529c38107fe7b6fdbbd39a662e" exitCode=0
Jul 08 07:08:51.841236 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:51.841239    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerDied","Data":"24ef4dd0f6c83b99be1ef5114a33d1d4428355529c38107fe7b6fdbbd39a662e"}
Jul 08 07:08:51.841800 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:51.841288    2571 scope.go:117] "RemoveContainer" containerID="097d4f6618c1fa410137b011648dbccf8bf4580fa88eecf18599a535ec157b9a"
Jul 08 07:08:51.841800 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:51.841711    2571 scope.go:117] "RemoveContainer" containerID="24ef4dd0f6c83b99be1ef5114a33d1d4428355529c38107fe7b6fdbbd39a662e"
Jul 08 07:08:52.003045 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.003001    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:52.003196 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.003089    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:52.003196 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.003134    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:08:52.849181 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.849147    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerStarted","Data":"a17ddf11ac1e6cdea0df6bde69da0aefb4ece25ed5cf27f574352bb5b5e1cbf4"}
Jul 08 07:08:52.851311 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.851292    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj_919f405b-87cd-4f78-9b92-a6a400dfbaa9/main/0.log"
Jul 08 07:08:52.851737 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.851713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerStarted","Data":"5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731"}
Jul 08 07:08:52.851978 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:52.851963    2571 scope.go:117] "RemoveContainer" containerID="faac7d276ea7ed9f6a807574eaecda9195f44d9ca2e748ab7afff6a5f6eac11f"
Jul 08 07:08:53.550754 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.550710    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:53.554956 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.554924    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:53.859034 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.859002    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj_919f405b-87cd-4f78-9b92-a6a400dfbaa9/main/0.log"
Jul 08 07:08:53.859575 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.859544    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerStarted","Data":"3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"}
Jul 08 07:08:53.860095 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.860066    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:53.860221 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.860128    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:53.860788 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.860772    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:08:53.883247 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.883201    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" podStartSLOduration=3.6330992 podStartE2EDuration="40.883186555s" totalImagesPullingTime="36.411993411s" totalInitContainerRuntime="838.093944ms" isStatefulPod=false podCreationTimestamp="2026-07-08 07:08:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:08:53.880194706 +0000 UTC m=+861.479484420" watchObservedRunningTime="2026-07-08 07:08:53.883186555 +0000 UTC m=+861.482476300"
Jul 08 07:08:53.917301 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.917266    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:08:53.917483 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:08:53.917358    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:09:03.918522 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:09:03.918486    2571 logging.go:55] [core] [Channel #17 SubChannel #18]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.36:9003", ServerName: "10.132.0.36:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.36:9003: connect: connection refused"
Jul 08 07:09:03.920115 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:03.920093    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:09:03.921315 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:03.921299    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:09:04.918277 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:04.918239    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.132.0.36:9003\" within 1s: context deadline exceeded"
Jul 08 07:09:24.915132 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:24.915055    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:09:26.310041 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:26.309996    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"]
Jul 08 07:09:26.312268 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:26.312241    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"]
Jul 08 07:09:26.312479 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:26.312454    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="tokenizer" containerID="cri-o://5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731" gracePeriod=60
Jul 08 07:09:26.323153 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:26.323115    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 07:09:26.323153 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:09:26.323153 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:09:26.323153 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696" cmd=["/bin/sleep","15"]
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:26.323179    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" message="" err=<
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:26.323194    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main" containerID="cri-o://3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"
Jul 08 07:09:26.323402 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:26.323234    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main" containerID="cri-o://3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696" gracePeriod=60
Jul 08 07:09:27.034089 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.034061    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj_919f405b-87cd-4f78-9b92-a6a400dfbaa9/main/0.log"
Jul 08 07:09:27.034467 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.034446    2571 generic.go:356] "Generic (PLEG): container finished" podID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerID="3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696" exitCode=0
Jul 08 07:09:27.034529 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.034512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerDied","Data":"3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"}
Jul 08 07:09:27.034583 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.034573    2571 scope.go:117] "RemoveContainer" containerID="faac7d276ea7ed9f6a807574eaecda9195f44d9ca2e748ab7afff6a5f6eac11f"
Jul 08 07:09:27.855470 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.855445    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:09:27.936827 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.936748    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gztw\" (UniqueName: \"kubernetes.io/projected/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kube-api-access-2gztw\") pod \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") "
Jul 08 07:09:27.936827 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.936827    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tls-certs\") pod \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") "
Jul 08 07:09:27.937055 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.936845    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-tmp\") pod \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") "
Jul 08 07:09:27.937055 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.936866    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-cache\") pod \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") "
Jul 08 07:09:27.937055 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.936904    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kserve-provision-location\") pod \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") "
Jul 08 07:09:27.937055 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.936932    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-uds\") pod \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\" (UID: \"919f405b-87cd-4f78-9b92-a6a400dfbaa9\") "
Jul 08 07:09:27.937264 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.937237    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "919f405b-87cd-4f78-9b92-a6a400dfbaa9" (UID: "919f405b-87cd-4f78-9b92-a6a400dfbaa9"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:27.937304 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.937245    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "919f405b-87cd-4f78-9b92-a6a400dfbaa9" (UID: "919f405b-87cd-4f78-9b92-a6a400dfbaa9"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:27.937304 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.937275    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "919f405b-87cd-4f78-9b92-a6a400dfbaa9" (UID: "919f405b-87cd-4f78-9b92-a6a400dfbaa9"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:27.937578 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.937555    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "919f405b-87cd-4f78-9b92-a6a400dfbaa9" (UID: "919f405b-87cd-4f78-9b92-a6a400dfbaa9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:27.939098 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.939072    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kube-api-access-2gztw" (OuterVolumeSpecName: "kube-api-access-2gztw") pod "919f405b-87cd-4f78-9b92-a6a400dfbaa9" (UID: "919f405b-87cd-4f78-9b92-a6a400dfbaa9"). InnerVolumeSpecName "kube-api-access-2gztw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:09:27.939192 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:27.939131    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "919f405b-87cd-4f78-9b92-a6a400dfbaa9" (UID: "919f405b-87cd-4f78-9b92-a6a400dfbaa9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:09:28.038446 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.038415    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:28.038629 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.038578    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gztw\" (UniqueName: \"kubernetes.io/projected/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kube-api-access-2gztw\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:28.038629 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.038598    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:28.038629 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.038613    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-tmp\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:28.038629 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.038625    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-tokenizer-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:28.038849 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.038637    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/919f405b-87cd-4f78-9b92-a6a400dfbaa9-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:28.042644 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.042619    2571 generic.go:356] "Generic (PLEG): container finished" podID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerID="5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731" exitCode=0
Jul 08 07:09:28.042796 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.042677    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"
Jul 08 07:09:28.042796 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.042731    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerDied","Data":"5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731"}
Jul 08 07:09:28.042877 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.042801    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj" event={"ID":"919f405b-87cd-4f78-9b92-a6a400dfbaa9","Type":"ContainerDied","Data":"e55167cb49286718a1114d1e29474ad8ea2075b4584b009c50df72f583c7604a"}
Jul 08 07:09:28.042877 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.042827    2571 scope.go:117] "RemoveContainer" containerID="3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"
Jul 08 07:09:28.053216 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.053201    2571 scope.go:117] "RemoveContainer" containerID="5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731"
Jul 08 07:09:28.061834 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.061818    2571 scope.go:117] "RemoveContainer" containerID="d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d"
Jul 08 07:09:28.066647 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.066625    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"]
Jul 08 07:09:28.070462 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.070443    2571 scope.go:117] "RemoveContainer" containerID="3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"
Jul 08 07:09:28.070742 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:28.070723    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696\": container with ID starting with 3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696 not found: ID does not exist" containerID="3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"
Jul 08 07:09:28.070840 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.070754    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696"} err="failed to get container status \"3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696\": rpc error: code = NotFound desc = could not find container \"3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696\": container with ID starting with 3539eff8e1e5b556e8204f73d9360e07f0b3d3303102cee77c915cbd61cc4696 not found: ID does not exist"
Jul 08 07:09:28.070840 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.070780    2571 scope.go:117] "RemoveContainer" containerID="5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731"
Jul 08 07:09:28.071081 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.071065    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5557946d8mdqj"]
Jul 08 07:09:28.071123 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:28.071064    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731\": container with ID starting with 5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731 not found: ID does not exist" containerID="5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731"
Jul 08 07:09:28.071156 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.071116    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731"} err="failed to get container status \"5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731\": rpc error: code = NotFound desc = could not find container \"5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731\": container with ID starting with 5b632e34b0848679ed6336f38f86f097bcd905d20d66b9ea9ad00beb36aba731 not found: ID does not exist"
Jul 08 07:09:28.071156 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.071132    2571 scope.go:117] "RemoveContainer" containerID="d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d"
Jul 08 07:09:28.071371 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:28.071353    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d\": container with ID starting with d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d not found: ID does not exist" containerID="d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d"
Jul 08 07:09:28.071410 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.071377    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d"} err="failed to get container status \"d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d\": rpc error: code = NotFound desc = could not find container \"d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d\": container with ID starting with d935b2348345c7dd3c30419880d0a9e3a51d7e116400c973002b82aaed4de48d not found: ID does not exist"
Jul 08 07:09:28.914283 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:28.914246    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" path="/var/lib/kubelet/pods/919f405b-87cd-4f78-9b92-a6a400dfbaa9/volumes"
Jul 08 07:09:41.344295 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.344239    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerName="main" containerID="cri-o://335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e" gracePeriod=45
Jul 08 07:09:41.603346 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.603323    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:09:41.759056 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759027    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxbdj\" (UniqueName: \"kubernetes.io/projected/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kube-api-access-rxbdj\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759056 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759064    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tmp-dir\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759290 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759114    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-home\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759290 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759141    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-dshm\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759290 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759172    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-model-cache\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759290 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759210    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kserve-provision-location\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759504 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759339    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tls-certs\") pod \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\" (UID: \"0766b965-2796-4e86-9af3-f2fe8ba2b8ee\") "
Jul 08 07:09:41.759504 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759362    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:41.759504 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759476    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:41.759725 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759679    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-model-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:41.759799 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759737    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tmp-dir\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:41.759799 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.759400    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-home" (OuterVolumeSpecName: "home") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:41.761460 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.761435    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kube-api-access-rxbdj" (OuterVolumeSpecName: "kube-api-access-rxbdj") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "kube-api-access-rxbdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:09:41.761575 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.761480    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:09:41.761575 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.761487    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-dshm" (OuterVolumeSpecName: "dshm") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:41.814806 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.814666    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0766b965-2796-4e86-9af3-f2fe8ba2b8ee" (UID: "0766b965-2796-4e86-9af3-f2fe8ba2b8ee"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:09:41.860591 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.860519    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxbdj\" (UniqueName: \"kubernetes.io/projected/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kube-api-access-rxbdj\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:41.860591 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.860545    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-home\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:41.860591 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.860560    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-dshm\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:41.860591 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.860571    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:41.860591 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:41.860586    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0766b965-2796-4e86-9af3-f2fe8ba2b8ee-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:09:42.111708 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.111598    2571 generic.go:356] "Generic (PLEG): container finished" podID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerID="335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e" exitCode=0
Jul 08 07:09:42.111708 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.111654    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"
Jul 08 07:09:42.111708 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.111653    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" event={"ID":"0766b965-2796-4e86-9af3-f2fe8ba2b8ee","Type":"ContainerDied","Data":"335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e"}
Jul 08 07:09:42.111982 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.111751    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz" event={"ID":"0766b965-2796-4e86-9af3-f2fe8ba2b8ee","Type":"ContainerDied","Data":"c15a961d809a4151983647a6d5a182073ebd289cb21a8e1e1bb0464d3a2dca0c"}
Jul 08 07:09:42.111982 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.111776    2571 scope.go:117] "RemoveContainer" containerID="335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e"
Jul 08 07:09:42.121282 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.121266    2571 scope.go:117] "RemoveContainer" containerID="35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4"
Jul 08 07:09:42.129963 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.129944    2571 scope.go:117] "RemoveContainer" containerID="335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e"
Jul 08 07:09:42.130223 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:42.130205    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e\": container with ID starting with 335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e not found: ID does not exist" containerID="335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e"
Jul 08 07:09:42.130274 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.130230    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e"} err="failed to get container status \"335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e\": rpc error: code = NotFound desc = could not find container \"335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e\": container with ID starting with 335055f179b10f1ee6e344593416331420583389f8137cf19a8367dbb1f4e44e not found: ID does not exist"
Jul 08 07:09:42.130274 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.130245    2571 scope.go:117] "RemoveContainer" containerID="35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4"
Jul 08 07:09:42.130451 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:09:42.130436    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4\": container with ID starting with 35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4 not found: ID does not exist" containerID="35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4"
Jul 08 07:09:42.130490 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.130455    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4"} err="failed to get container status \"35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4\": rpc error: code = NotFound desc = could not find container \"35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4\": container with ID starting with 35af80e3078d3a63ffa0ab7ec4b9884ef3d998b496492e4968a99fc315c59fc4 not found: ID does not exist"
Jul 08 07:09:42.133537 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.133517    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"]
Jul 08 07:09:42.136826 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.136807    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6484944dcc-s5znz"]
Jul 08 07:09:42.913835 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:09:42.913806    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" path="/var/lib/kubelet/pods/0766b965-2796-4e86-9af3-f2fe8ba2b8ee/volumes"
Jul 08 07:10:05.883417 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.883373    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"]
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.883946    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="tokenizer"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.883967    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="tokenizer"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.883982    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="storage-initializer"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.883992    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="storage-initializer"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884002    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884011    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884025    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerName="storage-initializer"
Jul 08 07:10:05.884042 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884032    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerName="storage-initializer"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884050    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884058    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884070    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerName="main"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884078    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerName="main"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884154    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884167    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="tokenizer"
Jul 08 07:10:05.884450 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.884180    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0766b965-2796-4e86-9af3-f2fe8ba2b8ee" containerName="main"
Jul 08 07:10:05.887309 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.887288    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:05.891065 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.891045    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Jul 08 07:10:05.891155 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.891102    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-psjvw\""
Jul 08 07:10:05.896288 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.896264    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"]
Jul 08 07:10:05.954207 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.954175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:05.954373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.954248    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:05.954373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.954274    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:05.954373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.954302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw975\" (UniqueName: \"kubernetes.io/projected/7335c58b-93f0-49b3-aad0-ae3ee22ed037-kube-api-access-lw975\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:05.954373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.954323    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:05.954603 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:05.954447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.055901 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.055863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056079 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.055930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056079 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.055956    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056079 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.055980    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw975\" (UniqueName: \"kubernetes.io/projected/7335c58b-93f0-49b3-aad0-ae3ee22ed037-kube-api-access-lw975\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056079 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.056009    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056273 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.056101    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056344 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.056322    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056407 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.056372    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.056530 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.056511    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.058403 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.058382    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.058545 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.058528    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.064600 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.064579    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw975\" (UniqueName: \"kubernetes.io/projected/7335c58b-93f0-49b3-aad0-ae3ee22ed037-kube-api-access-lw975\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.197933 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.197852    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:06.346207 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:06.346181    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"]
Jul 08 07:10:06.348493 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:10:06.348463    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7335c58b_93f0_49b3_aad0_ae3ee22ed037.slice/crio-ca678c78f537c568a4b4d0fe22ca2285c39794fec0a1f73164c6de653b6d170c WatchSource:0}: Error finding container ca678c78f537c568a4b4d0fe22ca2285c39794fec0a1f73164c6de653b6d170c: Status 404 returned error can't find the container with id ca678c78f537c568a4b4d0fe22ca2285c39794fec0a1f73164c6de653b6d170c
Jul 08 07:10:07.234702 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:07.234647    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8" event={"ID":"7335c58b-93f0-49b3-aad0-ae3ee22ed037","Type":"ContainerStarted","Data":"07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f"}
Jul 08 07:10:07.235099 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:07.234715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8" event={"ID":"7335c58b-93f0-49b3-aad0-ae3ee22ed037","Type":"ContainerStarted","Data":"ca678c78f537c568a4b4d0fe22ca2285c39794fec0a1f73164c6de653b6d170c"}
Jul 08 07:10:07.250586 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:07.250531    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8" podStartSLOduration=2.250516968 podStartE2EDuration="2.250516968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:10:07.249482382 +0000 UTC m=+934.848772085" watchObservedRunningTime="2026-07-08 07:10:07.250516968 +0000 UTC m=+934.849806670"
Jul 08 07:10:08.198006 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:08.197979    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:16.198678 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:16.198640    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:16.211426 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:16.211396    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:16.287341 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:16.287311    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:40.412532 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:40.412492    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"]
Jul 08 07:10:55.433917 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.433822    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8" podUID="7335c58b-93f0-49b3-aad0-ae3ee22ed037" containerName="main" containerID="cri-o://07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f" gracePeriod=45
Jul 08 07:10:55.697107 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.697050    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:55.708129 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708109    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-home\") pod \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") "
Jul 08 07:10:55.708256 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708140    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-dshm\") pod \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") "
Jul 08 07:10:55.708256 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708194    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tmp-dir\") pod \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") "
Jul 08 07:10:55.708373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708258    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-model-cache\") pod \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") "
Jul 08 07:10:55.708373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708282    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tls-certs\") pod \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") "
Jul 08 07:10:55.708373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708327    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw975\" (UniqueName: \"kubernetes.io/projected/7335c58b-93f0-49b3-aad0-ae3ee22ed037-kube-api-access-lw975\") pod \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\" (UID: \"7335c58b-93f0-49b3-aad0-ae3ee22ed037\") "
Jul 08 07:10:55.708520 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708397    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-home" (OuterVolumeSpecName: "home") pod "7335c58b-93f0-49b3-aad0-ae3ee22ed037" (UID: "7335c58b-93f0-49b3-aad0-ae3ee22ed037"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:10:55.708520 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708424    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7335c58b-93f0-49b3-aad0-ae3ee22ed037" (UID: "7335c58b-93f0-49b3-aad0-ae3ee22ed037"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:10:55.708520 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708506    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-model-cache" (OuterVolumeSpecName: "model-cache") pod "7335c58b-93f0-49b3-aad0-ae3ee22ed037" (UID: "7335c58b-93f0-49b3-aad0-ae3ee22ed037"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:10:55.708680 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708623    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-home\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:10:55.708680 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708643    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tmp-dir\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:10:55.708680 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.708655    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-model-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:10:55.710306 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.710282    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-dshm" (OuterVolumeSpecName: "dshm") pod "7335c58b-93f0-49b3-aad0-ae3ee22ed037" (UID: "7335c58b-93f0-49b3-aad0-ae3ee22ed037"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:10:55.710451 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.710432    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7335c58b-93f0-49b3-aad0-ae3ee22ed037-kube-api-access-lw975" (OuterVolumeSpecName: "kube-api-access-lw975") pod "7335c58b-93f0-49b3-aad0-ae3ee22ed037" (UID: "7335c58b-93f0-49b3-aad0-ae3ee22ed037"). InnerVolumeSpecName "kube-api-access-lw975". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:10:55.710508 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.710463    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7335c58b-93f0-49b3-aad0-ae3ee22ed037" (UID: "7335c58b-93f0-49b3-aad0-ae3ee22ed037"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:10:55.810051 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.810022    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7335c58b-93f0-49b3-aad0-ae3ee22ed037-dshm\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:10:55.810051 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.810046    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7335c58b-93f0-49b3-aad0-ae3ee22ed037-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:10:55.810051 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:55.810057    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw975\" (UniqueName: \"kubernetes.io/projected/7335c58b-93f0-49b3-aad0-ae3ee22ed037-kube-api-access-lw975\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:10:56.475052 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.475018    2571 generic.go:356] "Generic (PLEG): container finished" podID="7335c58b-93f0-49b3-aad0-ae3ee22ed037" containerID="07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f" exitCode=0
Jul 08 07:10:56.475509 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.475069    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"
Jul 08 07:10:56.475509 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.475112    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8" event={"ID":"7335c58b-93f0-49b3-aad0-ae3ee22ed037","Type":"ContainerDied","Data":"07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f"}
Jul 08 07:10:56.475509 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.475176    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8" event={"ID":"7335c58b-93f0-49b3-aad0-ae3ee22ed037","Type":"ContainerDied","Data":"ca678c78f537c568a4b4d0fe22ca2285c39794fec0a1f73164c6de653b6d170c"}
Jul 08 07:10:56.475509 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.475210    2571 scope.go:117] "RemoveContainer" containerID="07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f"
Jul 08 07:10:56.484522 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.484507    2571 scope.go:117] "RemoveContainer" containerID="07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f"
Jul 08 07:10:56.484793 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:10:56.484776    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f\": container with ID starting with 07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f not found: ID does not exist" containerID="07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f"
Jul 08 07:10:56.484863 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.484806    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f"} err="failed to get container status \"07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f\": rpc error: code = NotFound desc = could not find container \"07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f\": container with ID starting with 07d6e8e11b43930c8d96260c6e0ab4027fbee548558e5ad46c177b27aa3a8f6f not found: ID does not exist"
Jul 08 07:10:56.496312 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.496289    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"]
Jul 08 07:10:56.499497 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.499476    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879xhjx8"]
Jul 08 07:10:56.913802 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:10:56.913771    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7335c58b-93f0-49b3-aad0-ae3ee22ed037" path="/var/lib/kubelet/pods/7335c58b-93f0-49b3-aad0-ae3ee22ed037/volumes"
Jul 08 07:11:07.686985 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.686951    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"]
Jul 08 07:11:07.687419 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.687313    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7335c58b-93f0-49b3-aad0-ae3ee22ed037" containerName="main"
Jul 08 07:11:07.687419 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.687324    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7335c58b-93f0-49b3-aad0-ae3ee22ed037" containerName="main"
Jul 08 07:11:07.687419 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.687392    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7335c58b-93f0-49b3-aad0-ae3ee22ed037" containerName="main"
Jul 08 07:11:07.687419 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.687400    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="919f405b-87cd-4f78-9b92-a6a400dfbaa9" containerName="main"
Jul 08 07:11:07.690296 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.690277    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.694283 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.694205    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-psjvw\""
Jul 08 07:11:07.694283 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.694210    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Jul 08 07:11:07.697521 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.697496    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"]
Jul 08 07:11:07.819653 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.819618    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.819653 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.819650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.819886 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.819674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.819886 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.819715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.819886 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.819791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mnjh\" (UniqueName: \"kubernetes.io/projected/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-kube-api-access-8mnjh\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.819886 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.819865    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.920656 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.920625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.920808 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.920677    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.920808 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.920719    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.920808 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.920755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.920808 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.920790    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.921040 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.920943    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mnjh\" (UniqueName: \"kubernetes.io/projected/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-kube-api-access-8mnjh\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.921156 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.921128    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.921343 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.921321    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.921501 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.921403    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.923230 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.923204    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.923343 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.923327    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:07.936800 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:07.936777    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mnjh\" (UniqueName: \"kubernetes.io/projected/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-kube-api-access-8mnjh\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:08.003146 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:08.003082    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:08.144649 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:08.144621    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"]
Jul 08 07:11:08.146505 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:11:08.146476    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc12aaf_2832_4433_8fd9_4cd57b596a2a.slice/crio-c44e43026b56753c0b06a811d40f69d43d43cc687e58f5928f9fcdfd7dd52c9b WatchSource:0}: Error finding container c44e43026b56753c0b06a811d40f69d43d43cc687e58f5928f9fcdfd7dd52c9b: Status 404 returned error can't find the container with id c44e43026b56753c0b06a811d40f69d43d43cc687e58f5928f9fcdfd7dd52c9b
Jul 08 07:11:08.532328 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:08.532296    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz" event={"ID":"1cc12aaf-2832-4433-8fd9-4cd57b596a2a","Type":"ContainerStarted","Data":"babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4"}
Jul 08 07:11:08.532502 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:08.532343    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz" event={"ID":"1cc12aaf-2832-4433-8fd9-4cd57b596a2a","Type":"ContainerStarted","Data":"c44e43026b56753c0b06a811d40f69d43d43cc687e58f5928f9fcdfd7dd52c9b"}
Jul 08 07:11:08.550965 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:08.550911    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz" podStartSLOduration=1.550895546 podStartE2EDuration="1.550895546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:11:08.548107686 +0000 UTC m=+996.147397391" watchObservedRunningTime="2026-07-08 07:11:08.550895546 +0000 UTC m=+996.150185249"
Jul 08 07:11:09.004130 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:09.004098    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:18.004062 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:18.004011    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:18.016973 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:18.016940    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:18.591391 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:18.591364    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:11:45.401660 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.401621    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"]
Jul 08 07:11:45.406141 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.406125    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.408823 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.408800    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Jul 08 07:11:45.409936 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.409919    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa-dockercfg-9hmmk\""
Jul 08 07:11:45.414396 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.414378    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"]
Jul 08 07:11:45.463370 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.463338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1c370576-60be-418f-a7d4-006bb3bd04ca-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.463545 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.463403    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c370576-60be-418f-a7d4-006bb3bd04ca-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.463545 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.463488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-585n7\" (UniqueName: \"kubernetes.io/projected/1c370576-60be-418f-a7d4-006bb3bd04ca-kube-api-access-585n7\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.564566 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.564530    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-585n7\" (UniqueName: \"kubernetes.io/projected/1c370576-60be-418f-a7d4-006bb3bd04ca-kube-api-access-585n7\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.564737 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.564573    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1c370576-60be-418f-a7d4-006bb3bd04ca-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.564737 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.564650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c370576-60be-418f-a7d4-006bb3bd04ca-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.565089 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.565065    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1c370576-60be-418f-a7d4-006bb3bd04ca-tokenizer-uds\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.567488 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.567461    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c370576-60be-418f-a7d4-006bb3bd04ca-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.572752 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.572725    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-585n7\" (UniqueName: \"kubernetes.io/projected/1c370576-60be-418f-a7d4-006bb3bd04ca-kube-api-access-585n7\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") " pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.716253 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.716166    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:45.855546 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:45.855521    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"]
Jul 08 07:11:45.858004 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:11:45.857971    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c370576_60be_418f_a7d4_006bb3bd04ca.slice/crio-c73dcaa7355621eb00420e2c65e6bb62d6670044afbfb40ec1fe74221ff4bc31 WatchSource:0}: Error finding container c73dcaa7355621eb00420e2c65e6bb62d6670044afbfb40ec1fe74221ff4bc31: Status 404 returned error can't find the container with id c73dcaa7355621eb00420e2c65e6bb62d6670044afbfb40ec1fe74221ff4bc31
Jul 08 07:11:46.729440 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:46.729401    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" event={"ID":"1c370576-60be-418f-a7d4-006bb3bd04ca","Type":"ContainerStarted","Data":"f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"}
Jul 08 07:11:46.729867 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:46.729461    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" event={"ID":"1c370576-60be-418f-a7d4-006bb3bd04ca","Type":"ContainerStarted","Data":"c73dcaa7355621eb00420e2c65e6bb62d6670044afbfb40ec1fe74221ff4bc31"}
Jul 08 07:11:46.729867 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:46.729569    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:11:46.749298 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:11:46.749255    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" podStartSLOduration=1.7492412659999999 podStartE2EDuration="1.749241266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:11:46.748650186 +0000 UTC m=+1034.347939900" watchObservedRunningTime="2026-07-08 07:11:46.749241266 +0000 UTC m=+1034.348530969"
Jul 08 07:12:17.737523 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:17.737496    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:12:34.491010 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.490914    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"]
Jul 08 07:12:34.501010 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:12:34.500976    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 07:12:34.501010 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:12:34.501010 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:12:34.501010 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50" cmd=["/bin/sleep","15"]
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.501026    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" message="" err=<
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:12:34.501038    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" podUID="1c370576-60be-418f-a7d4-006bb3bd04ca" containerName="main" containerID="cri-o://f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"
Jul 08 07:12:34.501300 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.501084    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" podUID="1c370576-60be-418f-a7d4-006bb3bd04ca" containerName="main" containerID="cri-o://f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50" gracePeriod=60
Jul 08 07:12:34.752952 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.752891    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:12:34.818183 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.818153    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c370576-60be-418f-a7d4-006bb3bd04ca-tls-certs\") pod \"1c370576-60be-418f-a7d4-006bb3bd04ca\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") "
Jul 08 07:12:34.818330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.818225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1c370576-60be-418f-a7d4-006bb3bd04ca-tokenizer-uds\") pod \"1c370576-60be-418f-a7d4-006bb3bd04ca\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") "
Jul 08 07:12:34.818330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.818315    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-585n7\" (UniqueName: \"kubernetes.io/projected/1c370576-60be-418f-a7d4-006bb3bd04ca-kube-api-access-585n7\") pod \"1c370576-60be-418f-a7d4-006bb3bd04ca\" (UID: \"1c370576-60be-418f-a7d4-006bb3bd04ca\") "
Jul 08 07:12:34.818501 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.818478    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c370576-60be-418f-a7d4-006bb3bd04ca-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "1c370576-60be-418f-a7d4-006bb3bd04ca" (UID: "1c370576-60be-418f-a7d4-006bb3bd04ca"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:12:34.820441 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.820413    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c370576-60be-418f-a7d4-006bb3bd04ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1c370576-60be-418f-a7d4-006bb3bd04ca" (UID: "1c370576-60be-418f-a7d4-006bb3bd04ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:12:34.820560 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.820415    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c370576-60be-418f-a7d4-006bb3bd04ca-kube-api-access-585n7" (OuterVolumeSpecName: "kube-api-access-585n7") pod "1c370576-60be-418f-a7d4-006bb3bd04ca" (UID: "1c370576-60be-418f-a7d4-006bb3bd04ca"). InnerVolumeSpecName "kube-api-access-585n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:12:34.918960 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.918933    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1c370576-60be-418f-a7d4-006bb3bd04ca-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:12:34.918960 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.918958    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-585n7\" (UniqueName: \"kubernetes.io/projected/1c370576-60be-418f-a7d4-006bb3bd04ca-kube-api-access-585n7\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:12:34.919141 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.918971    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c370576-60be-418f-a7d4-006bb3bd04ca-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:12:34.973391 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.973360    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c370576-60be-418f-a7d4-006bb3bd04ca" containerID="f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50" exitCode=0
Jul 08 07:12:34.973548 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.973403    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"
Jul 08 07:12:34.973548 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.973427    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" event={"ID":"1c370576-60be-418f-a7d4-006bb3bd04ca","Type":"ContainerDied","Data":"f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"}
Jul 08 07:12:34.973548 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.973467    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p" event={"ID":"1c370576-60be-418f-a7d4-006bb3bd04ca","Type":"ContainerDied","Data":"c73dcaa7355621eb00420e2c65e6bb62d6670044afbfb40ec1fe74221ff4bc31"}
Jul 08 07:12:34.973548 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.973488    2571 scope.go:117] "RemoveContainer" containerID="f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"
Jul 08 07:12:34.982866 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.982764    2571 scope.go:117] "RemoveContainer" containerID="f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"
Jul 08 07:12:34.983071 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:12:34.983038    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50\": container with ID starting with f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50 not found: ID does not exist" containerID="f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"
Jul 08 07:12:34.983115 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.983080    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50"} err="failed to get container status \"f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50\": rpc error: code = NotFound desc = could not find container \"f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50\": container with ID starting with f8963e3634941574d9664043de0e403b81267a139bae9ef2d1858ab1d9c56a50 not found: ID does not exist"
Jul 08 07:12:34.989955 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.989934    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"]
Jul 08 07:12:34.993980 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:34.993960    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-sche4284p"]
Jul 08 07:12:36.914969 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:36.914934    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c370576-60be-418f-a7d4-006bb3bd04ca" path="/var/lib/kubelet/pods/1c370576-60be-418f-a7d4-006bb3bd04ca/volumes"
Jul 08 07:12:53.662100 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:12:53.662066    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"]
Jul 08 07:13:08.690978 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:08.690926    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz" podUID="1cc12aaf-2832-4433-8fd9-4cd57b596a2a" containerName="main" containerID="cri-o://babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4" gracePeriod=45
Jul 08 07:13:08.947476 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:08.947415    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:13:09.028810 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.028774    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mnjh\" (UniqueName: \"kubernetes.io/projected/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-kube-api-access-8mnjh\") pod \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") "
Jul 08 07:13:09.028984 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.028830    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tls-certs\") pod \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") "
Jul 08 07:13:09.028984 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.028850    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-home\") pod \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") "
Jul 08 07:13:09.028984 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.028881    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-model-cache\") pod \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") "
Jul 08 07:13:09.028984 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.028898    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-dshm\") pod \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") "
Jul 08 07:13:09.029182 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.029133    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-home" (OuterVolumeSpecName: "home") pod "1cc12aaf-2832-4433-8fd9-4cd57b596a2a" (UID: "1cc12aaf-2832-4433-8fd9-4cd57b596a2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:13:09.029242 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.029191    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1cc12aaf-2832-4433-8fd9-4cd57b596a2a" (UID: "1cc12aaf-2832-4433-8fd9-4cd57b596a2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:13:09.031204 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.031173    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1cc12aaf-2832-4433-8fd9-4cd57b596a2a" (UID: "1cc12aaf-2832-4433-8fd9-4cd57b596a2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:13:09.031350 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.031215    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-kube-api-access-8mnjh" (OuterVolumeSpecName: "kube-api-access-8mnjh") pod "1cc12aaf-2832-4433-8fd9-4cd57b596a2a" (UID: "1cc12aaf-2832-4433-8fd9-4cd57b596a2a"). InnerVolumeSpecName "kube-api-access-8mnjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:13:09.031350 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.031219    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-dshm" (OuterVolumeSpecName: "dshm") pod "1cc12aaf-2832-4433-8fd9-4cd57b596a2a" (UID: "1cc12aaf-2832-4433-8fd9-4cd57b596a2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:13:09.129701 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.129650    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tmp-dir\") pod \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\" (UID: \"1cc12aaf-2832-4433-8fd9-4cd57b596a2a\") "
Jul 08 07:13:09.129972 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.129946    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1cc12aaf-2832-4433-8fd9-4cd57b596a2a" (UID: "1cc12aaf-2832-4433-8fd9-4cd57b596a2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:13:09.130029 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.129983    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:13:09.130029 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.130000    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-home\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:13:09.130029 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.130010    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-model-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:13:09.130029 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.130018    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-dshm\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:13:09.130029 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.130026    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mnjh\" (UniqueName: \"kubernetes.io/projected/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-kube-api-access-8mnjh\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:13:09.145120 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.145088    2571 generic.go:356] "Generic (PLEG): container finished" podID="1cc12aaf-2832-4433-8fd9-4cd57b596a2a" containerID="babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4" exitCode=0
Jul 08 07:13:09.145272 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.145135    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"
Jul 08 07:13:09.145272 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.145152    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz" event={"ID":"1cc12aaf-2832-4433-8fd9-4cd57b596a2a","Type":"ContainerDied","Data":"babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4"}
Jul 08 07:13:09.145272 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.145206    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz" event={"ID":"1cc12aaf-2832-4433-8fd9-4cd57b596a2a","Type":"ContainerDied","Data":"c44e43026b56753c0b06a811d40f69d43d43cc687e58f5928f9fcdfd7dd52c9b"}
Jul 08 07:13:09.145272 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.145235    2571 scope.go:117] "RemoveContainer" containerID="babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4"
Jul 08 07:13:09.154618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.154599    2571 scope.go:117] "RemoveContainer" containerID="babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4"
Jul 08 07:13:09.154899 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:13:09.154881    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4\": container with ID starting with babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4 not found: ID does not exist" containerID="babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4"
Jul 08 07:13:09.154952 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.154909    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4"} err="failed to get container status \"babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4\": rpc error: code = NotFound desc = could not find container \"babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4\": container with ID starting with babae1de54a5f63b94b2f78c33bf256113d61b244b9acea3df7303e2cef5f8d4 not found: ID does not exist"
Jul 08 07:13:09.166901 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.166877    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"]
Jul 08 07:13:09.169412 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.169391    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4rx4cz"]
Jul 08 07:13:09.231222 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:09.231142    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc12aaf-2832-4433-8fd9-4cd57b596a2a-tmp-dir\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:13:10.914138 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:10.914108    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc12aaf-2832-4433-8fd9-4cd57b596a2a" path="/var/lib/kubelet/pods/1cc12aaf-2832-4433-8fd9-4cd57b596a2a/volumes"
Jul 08 07:13:23.965735 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.965702    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"]
Jul 08 07:13:23.966163 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.966030    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c370576-60be-418f-a7d4-006bb3bd04ca" containerName="main"
Jul 08 07:13:23.966163 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.966040    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c370576-60be-418f-a7d4-006bb3bd04ca" containerName="main"
Jul 08 07:13:23.966163 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.966054    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc12aaf-2832-4433-8fd9-4cd57b596a2a" containerName="main"
Jul 08 07:13:23.966163 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.966059    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc12aaf-2832-4433-8fd9-4cd57b596a2a" containerName="main"
Jul 08 07:13:23.966163 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.966136    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc12aaf-2832-4433-8fd9-4cd57b596a2a" containerName="main"
Jul 08 07:13:23.966163 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.966151    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c370576-60be-418f-a7d4-006bb3bd04ca" containerName="main"
Jul 08 07:13:23.970378 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.970360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:23.974437 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.974418    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-psjvw\""
Jul 08 07:13:23.974531 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.974418    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Jul 08 07:13:23.988780 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:23.988760    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"]
Jul 08 07:13:24.059330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.059301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.059492 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.059345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9rlm\" (UniqueName: \"kubernetes.io/projected/57fcba24-ec0b-46a6-86e2-a0a5d397132e-kube-api-access-r9rlm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.059492 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.059423    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.059492 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.059459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.059492 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.059483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.059629 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.059537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160285 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160285 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160297    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160518 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160329    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160518 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160518 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160675 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160528    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9rlm\" (UniqueName: \"kubernetes.io/projected/57fcba24-ec0b-46a6-86e2-a0a5d397132e-kube-api-access-r9rlm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160853 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160817    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160977 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160872    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.160977 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.160901    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.162810 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.162780    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.163283 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.163261    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.171330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.171306    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9rlm\" (UniqueName: \"kubernetes.io/projected/57fcba24-ec0b-46a6-86e2-a0a5d397132e-kube-api-access-r9rlm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.281188 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.281095    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:24.300893 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.300864    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"]
Jul 08 07:13:24.305137 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.305116    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.315091 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.315066    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-4b931143-epp-sa-dockercfg-lj6xl\""
Jul 08 07:13:24.330987 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.330961    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"]
Jul 08 07:13:24.362444 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.362389    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msdwg\" (UniqueName: \"kubernetes.io/projected/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-kube-api-access-msdwg\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.362578 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.362523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.362578 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.362572    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.447529 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.447488    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"]
Jul 08 07:13:24.457568 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:13:24.457539    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57fcba24_ec0b_46a6_86e2_a0a5d397132e.slice/crio-21055ca698e428f791cfd16c3ee9b0ef731d0647327dfef92a9c829fd4318357 WatchSource:0}: Error finding container 21055ca698e428f791cfd16c3ee9b0ef731d0647327dfef92a9c829fd4318357: Status 404 returned error can't find the container with id 21055ca698e428f791cfd16c3ee9b0ef731d0647327dfef92a9c829fd4318357
Jul 08 07:13:24.464215 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.464107    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.464215 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.464159    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.464313 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.464227    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msdwg\" (UniqueName: \"kubernetes.io/projected/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-kube-api-access-msdwg\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.464810 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.464785    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.467024 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.467000    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.475275 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.475255    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msdwg\" (UniqueName: \"kubernetes.io/projected/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-kube-api-access-msdwg\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.614888 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.614857    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:24.775114 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:24.775085    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"]
Jul 08 07:13:24.782023 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:13:24.781998    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae31e8b9_0e56_4508_89ff_36a6c6fc28fb.slice/crio-431c3835f16c5dc5af054cf6348c4d98e99d20bdabc8e18bc70ba5c3f158261c WatchSource:0}: Error finding container 431c3835f16c5dc5af054cf6348c4d98e99d20bdabc8e18bc70ba5c3f158261c: Status 404 returned error can't find the container with id 431c3835f16c5dc5af054cf6348c4d98e99d20bdabc8e18bc70ba5c3f158261c
Jul 08 07:13:25.225519 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.225466    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" event={"ID":"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb","Type":"ContainerStarted","Data":"0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"}
Jul 08 07:13:25.225962 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.225537    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:13:25.225962 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.225561    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" event={"ID":"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb","Type":"ContainerStarted","Data":"431c3835f16c5dc5af054cf6348c4d98e99d20bdabc8e18bc70ba5c3f158261c"}
Jul 08 07:13:25.226971 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.226946    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" event={"ID":"57fcba24-ec0b-46a6-86e2-a0a5d397132e","Type":"ContainerStarted","Data":"7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3"}
Jul 08 07:13:25.227100 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.226987    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" event={"ID":"57fcba24-ec0b-46a6-86e2-a0a5d397132e","Type":"ContainerStarted","Data":"21055ca698e428f791cfd16c3ee9b0ef731d0647327dfef92a9c829fd4318357"}
Jul 08 07:13:25.258124 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.258084    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" podStartSLOduration=1.25807244 podStartE2EDuration="1.25807244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:13:25.257780521 +0000 UTC m=+1132.857070248" watchObservedRunningTime="2026-07-08 07:13:25.25807244 +0000 UTC m=+1132.857362139"
Jul 08 07:13:25.281434 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.281408    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:25.291898 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:25.291853    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" podStartSLOduration=2.2918394060000002 podStartE2EDuration="2.291839406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 07:13:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:13:25.289578363 +0000 UTC m=+1132.888868065" watchObservedRunningTime="2026-07-08 07:13:25.291839406 +0000 UTC m=+1132.891129127"
Jul 08 07:13:34.281939 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:34.281906    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:34.294247 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:34.294218    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:35.285676 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:35.285650    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:13:56.233419 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:13:56.233347    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:16:00.198069 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.198036    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"]
Jul 08 07:16:00.206344 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.206311    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"]
Jul 08 07:16:00.216134 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:16:00.216103    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 07:16:00.216134 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:16:00.216134 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:16:00.216134 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d" cmd=["/bin/sleep","15"]
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.216147    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" message="" err=<
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:16:00.216159    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" podUID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" containerName="main" containerID="cri-o://0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"
Jul 08 07:16:00.216443 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.216206    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" podUID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" containerName="main" containerID="cri-o://0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d" gracePeriod=60
Jul 08 07:16:00.474066 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.474042    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:16:00.601079 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.601046    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msdwg\" (UniqueName: \"kubernetes.io/projected/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-kube-api-access-msdwg\") pod \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") "
Jul 08 07:16:00.601263 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.601090    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tls-certs\") pod \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") "
Jul 08 07:16:00.601263 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.601232    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tokenizer-uds\") pod \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\" (UID: \"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb\") "
Jul 08 07:16:00.601509 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.601485    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" (UID: "ae31e8b9-0e56-4508-89ff-36a6c6fc28fb"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:16:00.603320 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.603296    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-kube-api-access-msdwg" (OuterVolumeSpecName: "kube-api-access-msdwg") pod "ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" (UID: "ae31e8b9-0e56-4508-89ff-36a6c6fc28fb"). InnerVolumeSpecName "kube-api-access-msdwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:16:00.603424 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.603348    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" (UID: "ae31e8b9-0e56-4508-89ff-36a6c6fc28fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:16:00.702865 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.702776    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:00.702865 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.702811    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msdwg\" (UniqueName: \"kubernetes.io/projected/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-kube-api-access-msdwg\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:00.702865 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.702823    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:00.984938 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.984852    2571 generic.go:356] "Generic (PLEG): container finished" podID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" containerID="0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d" exitCode=0
Jul 08 07:16:00.984938 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.984895    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"
Jul 08 07:16:00.985126 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.984935    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" event={"ID":"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb","Type":"ContainerDied","Data":"0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"}
Jul 08 07:16:00.985126 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.984981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr" event={"ID":"ae31e8b9-0e56-4508-89ff-36a6c6fc28fb","Type":"ContainerDied","Data":"431c3835f16c5dc5af054cf6348c4d98e99d20bdabc8e18bc70ba5c3f158261c"}
Jul 08 07:16:00.985126 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.985000    2571 scope.go:117] "RemoveContainer" containerID="0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"
Jul 08 07:16:00.993952 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.993933    2571 scope.go:117] "RemoveContainer" containerID="0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"
Jul 08 07:16:00.994214 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:16:00.994191    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d\": container with ID starting with 0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d not found: ID does not exist" containerID="0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"
Jul 08 07:16:00.994264 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:00.994226    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d"} err="failed to get container status \"0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d\": rpc error: code = NotFound desc = could not find container \"0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d\": container with ID starting with 0cacc33a3b449c649904639b19889ce4cb131e73235a2378b9d23c69445cf33d not found: ID does not exist"
Jul 08 07:16:01.010234 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:01.010207    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"]
Jul 08 07:16:01.026211 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:01.026185    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schewfhvr"]
Jul 08 07:16:02.913991 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:02.913960    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" path="/var/lib/kubelet/pods/ae31e8b9-0e56-4508-89ff-36a6c6fc28fb/volumes"
Jul 08 07:16:15.220027 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.219983    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" podUID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" containerName="main" containerID="cri-o://7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3" gracePeriod=45
Jul 08 07:16:15.275215 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.275177    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" podUID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": dial tcp 10.132.0.40:8000: connect: connection refused"
Jul 08 07:16:15.474534 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.474481    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:16:15.535590 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535564    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tmp-dir\") pod \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") "
Jul 08 07:16:15.535773 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535626    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9rlm\" (UniqueName: \"kubernetes.io/projected/57fcba24-ec0b-46a6-86e2-a0a5d397132e-kube-api-access-r9rlm\") pod \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") "
Jul 08 07:16:15.535773 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535669    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-dshm\") pod \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") "
Jul 08 07:16:15.535773 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535753    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tls-certs\") pod \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") "
Jul 08 07:16:15.535956 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535782    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-home\") pod \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") "
Jul 08 07:16:15.535956 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535827    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57fcba24-ec0b-46a6-86e2-a0a5d397132e" (UID: "57fcba24-ec0b-46a6-86e2-a0a5d397132e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:16:15.535956 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.535852    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-model-cache\") pod \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\" (UID: \"57fcba24-ec0b-46a6-86e2-a0a5d397132e\") "
Jul 08 07:16:15.536102 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.536015    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-home" (OuterVolumeSpecName: "home") pod "57fcba24-ec0b-46a6-86e2-a0a5d397132e" (UID: "57fcba24-ec0b-46a6-86e2-a0a5d397132e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:16:15.536145 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.536105    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-model-cache" (OuterVolumeSpecName: "model-cache") pod "57fcba24-ec0b-46a6-86e2-a0a5d397132e" (UID: "57fcba24-ec0b-46a6-86e2-a0a5d397132e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:16:15.536204 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.536175    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-home\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:15.536204 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.536187    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-model-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:15.536204 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.536198    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tmp-dir\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:15.537912 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.537889    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57fcba24-ec0b-46a6-86e2-a0a5d397132e" (UID: "57fcba24-ec0b-46a6-86e2-a0a5d397132e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 07:16:15.538328 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.538308    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-dshm" (OuterVolumeSpecName: "dshm") pod "57fcba24-ec0b-46a6-86e2-a0a5d397132e" (UID: "57fcba24-ec0b-46a6-86e2-a0a5d397132e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 07:16:15.538371 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.538327    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57fcba24-ec0b-46a6-86e2-a0a5d397132e-kube-api-access-r9rlm" (OuterVolumeSpecName: "kube-api-access-r9rlm") pod "57fcba24-ec0b-46a6-86e2-a0a5d397132e" (UID: "57fcba24-ec0b-46a6-86e2-a0a5d397132e"). InnerVolumeSpecName "kube-api-access-r9rlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:16:15.637160 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.637129    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57fcba24-ec0b-46a6-86e2-a0a5d397132e-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:15.637160 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.637157    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9rlm\" (UniqueName: \"kubernetes.io/projected/57fcba24-ec0b-46a6-86e2-a0a5d397132e-kube-api-access-r9rlm\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:15.637160 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:15.637168    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57fcba24-ec0b-46a6-86e2-a0a5d397132e-dshm\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:16:16.059669 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.059636    2571 generic.go:356] "Generic (PLEG): container finished" podID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" containerID="7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3" exitCode=0
Jul 08 07:16:16.059849 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.059710    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"
Jul 08 07:16:16.059849 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.059735    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" event={"ID":"57fcba24-ec0b-46a6-86e2-a0a5d397132e","Type":"ContainerDied","Data":"7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3"}
Jul 08 07:16:16.059849 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.059788    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf" event={"ID":"57fcba24-ec0b-46a6-86e2-a0a5d397132e","Type":"ContainerDied","Data":"21055ca698e428f791cfd16c3ee9b0ef731d0647327dfef92a9c829fd4318357"}
Jul 08 07:16:16.059849 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.059813    2571 scope.go:117] "RemoveContainer" containerID="7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3"
Jul 08 07:16:16.069515 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.069499    2571 scope.go:117] "RemoveContainer" containerID="7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3"
Jul 08 07:16:16.069811 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:16:16.069791    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3\": container with ID starting with 7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3 not found: ID does not exist" containerID="7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3"
Jul 08 07:16:16.069867 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.069822    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3"} err="failed to get container status \"7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3\": rpc error: code = NotFound desc = could not find container \"7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3\": container with ID starting with 7b8b5df8223fc36ea4cea76dfbf9db96fd284d6596b3eeeb7cb2b1531b7ca1b3 not found: ID does not exist"
Jul 08 07:16:16.085826 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.085804    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"]
Jul 08 07:16:16.093836 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.093817    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-w7zzf"]
Jul 08 07:16:16.913854 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:16.913823    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" path="/var/lib/kubelet/pods/57fcba24-ec0b-46a6-86e2-a0a5d397132e/volumes"
Jul 08 07:16:26.797431 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.797396    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"]
Jul 08 07:16:26.797885 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.797861    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" containerName="main"
Jul 08 07:16:26.798030 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.797906    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" containerName="main"
Jul 08 07:16:26.798030 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.797929    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" containerName="main"
Jul 08 07:16:26.798030 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.797937    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" containerName="main"
Jul 08 07:16:26.798201 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.798059    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae31e8b9-0e56-4508-89ff-36a6c6fc28fb" containerName="main"
Jul 08 07:16:26.798201 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.798076    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="57fcba24-ec0b-46a6-86e2-a0a5d397132e" containerName="main"
Jul 08 07:16:26.802971 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.802948    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:26.806962 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.806936    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 07:16:26.807078 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.806982    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-psjvw\""
Jul 08 07:16:26.807078 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.806938    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\""
Jul 08 07:16:26.807078 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.807035    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa-dockercfg-z28xr\""
Jul 08 07:16:26.817760 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.817737    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"]
Jul 08 07:16:26.935766 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.935728    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b24e8e6-7f75-4a57-a998-e2c566c96672-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:26.935950 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.935796    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7b24e8e6-7f75-4a57-a998-e2c566c96672-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:26.935950 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.935843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7b24e8e6-7f75-4a57-a998-e2c566c96672-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:26.935950 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:26.935922    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqttg\" (UniqueName: \"kubernetes.io/projected/7b24e8e6-7f75-4a57-a998-e2c566c96672-kube-api-access-bqttg\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.036829 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.036792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b24e8e6-7f75-4a57-a998-e2c566c96672-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.036829 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.036838    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7b24e8e6-7f75-4a57-a998-e2c566c96672-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.037095 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.036859    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7b24e8e6-7f75-4a57-a998-e2c566c96672-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.037095 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.036894    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqttg\" (UniqueName: \"kubernetes.io/projected/7b24e8e6-7f75-4a57-a998-e2c566c96672-kube-api-access-bqttg\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.037268 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.037241    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7b24e8e6-7f75-4a57-a998-e2c566c96672-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.037383 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.037303    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7b24e8e6-7f75-4a57-a998-e2c566c96672-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.039618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.039597    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b24e8e6-7f75-4a57-a998-e2c566c96672-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.045067 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.045047    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqttg\" (UniqueName: \"kubernetes.io/projected/7b24e8e6-7f75-4a57-a998-e2c566c96672-kube-api-access-bqttg\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z\" (UID: \"7b24e8e6-7f75-4a57-a998-e2c566c96672\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.115230 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.115207    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:27.260663 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:27.260635    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"]
Jul 08 07:16:27.262666 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:16:27.262638    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b24e8e6_7f75_4a57_a998_e2c566c96672.slice/crio-0c3b2b06a0d236baab1bdda358f043c933fa0e789ab677865db5cc2d9a9c6d0b WatchSource:0}: Error finding container 0c3b2b06a0d236baab1bdda358f043c933fa0e789ab677865db5cc2d9a9c6d0b: Status 404 returned error can't find the container with id 0c3b2b06a0d236baab1bdda358f043c933fa0e789ab677865db5cc2d9a9c6d0b
Jul 08 07:16:28.120535 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:28.120504    2571 generic.go:356] "Generic (PLEG): container finished" podID="7b24e8e6-7f75-4a57-a998-e2c566c96672" containerID="0b79466f079583e4489f430f394e5dd146d4e3928cb18385f31050fb847c0e84" exitCode=0
Jul 08 07:16:28.120881 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:28.120594    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z" event={"ID":"7b24e8e6-7f75-4a57-a998-e2c566c96672","Type":"ContainerDied","Data":"0b79466f079583e4489f430f394e5dd146d4e3928cb18385f31050fb847c0e84"}
Jul 08 07:16:28.120881 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:28.120642    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z" event={"ID":"7b24e8e6-7f75-4a57-a998-e2c566c96672","Type":"ContainerStarted","Data":"0c3b2b06a0d236baab1bdda358f043c933fa0e789ab677865db5cc2d9a9c6d0b"}
Jul 08 07:16:29.127306 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:29.127266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z" event={"ID":"7b24e8e6-7f75-4a57-a998-e2c566c96672","Type":"ContainerStarted","Data":"483ea1d8ba85297c14764a994bba78acac3bae045901455c6c5eecee98b7ceac"}
Jul 08 07:16:29.127804 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:29.127377    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:16:29.150061 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:16:29.150017    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z" podStartSLOduration=2.380673546 podStartE2EDuration="3.150003861s" totalImagesPullingTime="0s" totalInitContainerRuntime="769.330315ms" isStatefulPod=false podCreationTimestamp="2026-07-08 07:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:16:29.147595402 +0000 UTC m=+1316.746885107" watchObservedRunningTime="2026-07-08 07:16:29.150003861 +0000 UTC m=+1316.749293560"
Jul 08 07:17:00.135763 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:17:00.135737    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z"
Jul 08 07:18:28.742139 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:28.742031    2571 generic.go:356] "Generic (PLEG): container finished" podID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerID="a17ddf11ac1e6cdea0df6bde69da0aefb4ece25ed5cf27f574352bb5b5e1cbf4" exitCode=0
Jul 08 07:18:28.742139 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:28.742074    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerDied","Data":"a17ddf11ac1e6cdea0df6bde69da0aefb4ece25ed5cf27f574352bb5b5e1cbf4"}
Jul 08 07:18:28.742668 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:28.742139    2571 scope.go:117] "RemoveContainer" containerID="24ef4dd0f6c83b99be1ef5114a33d1d4428355529c38107fe7b6fdbbd39a662e"
Jul 08 07:18:28.742668 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:28.742501    2571 scope.go:117] "RemoveContainer" containerID="a17ddf11ac1e6cdea0df6bde69da0aefb4ece25ed5cf27f574352bb5b5e1cbf4"
Jul 08 07:18:28.742793 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:18:28.742754    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 20s restarting failed container=istio-proxy pod=router-gateway-1-openshift-default-799f46c59b-kwjm2_kserve-ci-e2e-test(f353d789-6bd9-4653-a007-ac3dab8cbbe6)\"" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6"
Jul 08 07:18:38.859927 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:38.859895    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:18:38.860319 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:38.860305    2571 scope.go:117] "RemoveContainer" containerID="a17ddf11ac1e6cdea0df6bde69da0aefb4ece25ed5cf27f574352bb5b5e1cbf4"
Jul 08 07:18:38.860501 ip-10-0-129-54 kubenswrapper[2571]: E0708 07:18:38.860482    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"istio-proxy\" with CrashLoopBackOff: \"back-off 20s restarting failed container=istio-proxy pod=router-gateway-1-openshift-default-799f46c59b-kwjm2_kserve-ci-e2e-test(f353d789-6bd9-4653-a007-ac3dab8cbbe6)\"" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6"
Jul 08 07:18:53.908628 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:53.908587    2571 scope.go:117] "RemoveContainer" containerID="a17ddf11ac1e6cdea0df6bde69da0aefb4ece25ed5cf27f574352bb5b5e1cbf4"
Jul 08 07:18:53.909968 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:53.909940    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:18:53.910054 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:53.910010    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:18:53.910054 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:53.910040    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 07:18:54.880491 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:54.880457    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" event={"ID":"f353d789-6bd9-4653-a007-ac3dab8cbbe6","Type":"ContainerStarted","Data":"88576e42cc05072fc0babc912dbca6a6d74bcbd77df996378b5ecda4c10a9d00"}
Jul 08 07:18:55.550581 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:55.550533    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:18:55.551711 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:55.551658    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:18:56.550465 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:56.550417    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:18:57.550719 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:57.550655    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:18:58.550483 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:58.550431    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:18:59.550822 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:18:59.550768    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2" podUID="f353d789-6bd9-4653-a007-ac3dab8cbbe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 08 07:19:00.554003 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:19:00.553969    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:19:00.554503 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:19:00.554254    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:19:00.554974 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:19:00.554954    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-kwjm2"
Jul 08 07:33:03.054067 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:03.054035    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 07:33:03.061609 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:03.061584    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 07:33:12.805355 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.805324    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"]
Jul 08 07:33:12.808828 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.808807    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:12.811325 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.811303    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Jul 08 07:33:12.811467 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.811302    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-gp76f\""
Jul 08 07:33:12.820618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.820589    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"]
Jul 08 07:33:12.970206 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.970163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgx84\" (UniqueName: \"kubernetes.io/projected/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-kube-api-access-xgx84\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:12.970379 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.970242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:12.970379 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.970279    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:12.970379 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:12.970350    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.071634 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.071547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.071634 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.071585    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.071857 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.071648    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.071857 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.071788    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgx84\" (UniqueName: \"kubernetes.io/projected/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-kube-api-access-xgx84\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.072058 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.072031    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-tokenizer-uds\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.072141 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.072125    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.074469 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.074448    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.080462 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.080437    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgx84\" (UniqueName: \"kubernetes.io/projected/685b6e6b-dedb-4dc0-ba22-2711b4b10f95-kube-api-access-xgx84\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4\" (UID: \"685b6e6b-dedb-4dc0-ba22-2711b4b10f95\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.119504 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.119468    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:13.267501 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.267472    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"]
Jul 08 07:33:13.268717 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:33:13.268668    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod685b6e6b_dedb_4dc0_ba22_2711b4b10f95.slice/crio-b88c6722f7739d307dc449d425d9caa5b08f61ad7a52c0b18bee9a1f68a97efd WatchSource:0}: Error finding container b88c6722f7739d307dc449d425d9caa5b08f61ad7a52c0b18bee9a1f68a97efd: Status 404 returned error can't find the container with id b88c6722f7739d307dc449d425d9caa5b08f61ad7a52c0b18bee9a1f68a97efd
Jul 08 07:33:13.273274 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:13.273249    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 07:33:14.143018 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:14.142975    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4" event={"ID":"685b6e6b-dedb-4dc0-ba22-2711b4b10f95","Type":"ContainerStarted","Data":"724c09503ef09af5db70189ab0c45a67124493884a5222fa8b8cc01d0a13448b"}
Jul 08 07:33:14.143414 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:14.143039    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4" event={"ID":"685b6e6b-dedb-4dc0-ba22-2711b4b10f95","Type":"ContainerStarted","Data":"b88c6722f7739d307dc449d425d9caa5b08f61ad7a52c0b18bee9a1f68a97efd"}
Jul 08 07:33:15.149333 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:15.149299    2571 generic.go:356] "Generic (PLEG): container finished" podID="685b6e6b-dedb-4dc0-ba22-2711b4b10f95" containerID="724c09503ef09af5db70189ab0c45a67124493884a5222fa8b8cc01d0a13448b" exitCode=0
Jul 08 07:33:15.149736 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:15.149393    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4" event={"ID":"685b6e6b-dedb-4dc0-ba22-2711b4b10f95","Type":"ContainerDied","Data":"724c09503ef09af5db70189ab0c45a67124493884a5222fa8b8cc01d0a13448b"}
Jul 08 07:33:16.157294 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:16.157255    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4" event={"ID":"685b6e6b-dedb-4dc0-ba22-2711b4b10f95","Type":"ContainerStarted","Data":"91a693fc1f7c846b143f5db47763719b0f2c6877c091707b900c7cadcc15f211"}
Jul 08 07:33:16.157815 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:16.157448    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:33:16.178618 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:16.178479    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4" podStartSLOduration=3.36618103 podStartE2EDuration="4.178458512s" totalImagesPullingTime="0s" totalInitContainerRuntime="812.277482ms" isStatefulPod=false podCreationTimestamp="2026-07-08 07:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:33:16.177995161 +0000 UTC m=+2323.777284871" watchObservedRunningTime="2026-07-08 07:33:16.178458512 +0000 UTC m=+2323.777748217"
Jul 08 07:33:47.164205 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:33:47.164172    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4"
Jul 08 07:36:38.531117 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:38.531037    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 07:36:38.539845 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:38.539822    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 07:36:42.799933 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:42.799891    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"]
Jul 08 07:36:42.803559 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:42.803535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:42.813324 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:42.813299    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"]
Jul 08 07:36:42.927995 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:42.927962    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wxhw\" (UniqueName: \"kubernetes.io/projected/5837851c-03f6-49c9-bce4-9ef14221d0a8-kube-api-access-9wxhw\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:43.028764 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:43.028731    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wxhw\" (UniqueName: \"kubernetes.io/projected/5837851c-03f6-49c9-bce4-9ef14221d0a8-kube-api-access-9wxhw\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:43.039050 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:43.039024    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wxhw\" (UniqueName: \"kubernetes.io/projected/5837851c-03f6-49c9-bce4-9ef14221d0a8-kube-api-access-9wxhw\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:51.719665 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:51.719626    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:51.725636 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:51.725610    2571 operation_generator.go:1469] "Controller attach succeeded for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") device path: \"\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:51.820836 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:51.820804    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:51.821035 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:51.820979    2571 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") DevicePath \"\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:51.824101 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:51.824078    2571 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") DevicePath \"csi-12508cac709fb80c00b6b231494ac5ab89dc7e7d0c604f2c45f16c875ec54220\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:52.041150 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:52.041110    2571 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/d8cc943a74105dc3dfe7dcd9ae9ba70ab2d5536dabcf5b22935bc499f06ab79b/globalmount\"" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:52.055714 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:52.055659    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"e2e-pvc-model-download-vfrk4\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") " pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:52.113547 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:52.113513    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:36:52.256373 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:52.256346    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"]
Jul 08 07:36:52.258279 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:36:52.258241    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5837851c_03f6_49c9_bce4_9ef14221d0a8.slice/crio-9e977e81b76681d074de74ac75e13adf122717d20013281c16433a0097edf846 WatchSource:0}: Error finding container 9e977e81b76681d074de74ac75e13adf122717d20013281c16433a0097edf846: Status 404 returned error can't find the container with id 9e977e81b76681d074de74ac75e13adf122717d20013281c16433a0097edf846
Jul 08 07:36:53.264854 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:53.264808    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4" event={"ID":"5837851c-03f6-49c9-bce4-9ef14221d0a8","Type":"ContainerStarted","Data":"53036fe7a3f252c8e663360db2467ff31efeb513728476f7e24d894ec23513f9"}
Jul 08 07:36:53.264854 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:53.264867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4" event={"ID":"5837851c-03f6-49c9-bce4-9ef14221d0a8","Type":"ContainerStarted","Data":"9e977e81b76681d074de74ac75e13adf122717d20013281c16433a0097edf846"}
Jul 08 07:36:53.299245 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:53.299174    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4" podStartSLOduration=14.299150629 podStartE2EDuration="14.299150629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 07:36:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:36:53.296287515 +0000 UTC m=+2540.895577218" watchObservedRunningTime="2026-07-08 07:36:53.299150629 +0000 UTC m=+2540.898440335"
Jul 08 07:36:59.296039 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:59.296001    2571 generic.go:356] "Generic (PLEG): container finished" podID="5837851c-03f6-49c9-bce4-9ef14221d0a8" containerID="53036fe7a3f252c8e663360db2467ff31efeb513728476f7e24d894ec23513f9" exitCode=0
Jul 08 07:36:59.296522 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:36:59.296073    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4" event={"ID":"5837851c-03f6-49c9-bce4-9ef14221d0a8","Type":"ContainerDied","Data":"53036fe7a3f252c8e663360db2467ff31efeb513728476f7e24d894ec23513f9"}
Jul 08 07:37:00.455762 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.455734    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:37:00.507339 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.507298    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-storage\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") pod \"5837851c-03f6-49c9-bce4-9ef14221d0a8\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") "
Jul 08 07:37:00.507511 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.507371    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wxhw\" (UniqueName: \"kubernetes.io/projected/5837851c-03f6-49c9-bce4-9ef14221d0a8-kube-api-access-9wxhw\") pod \"5837851c-03f6-49c9-bce4-9ef14221d0a8\" (UID: \"5837851c-03f6-49c9-bce4-9ef14221d0a8\") "
Jul 08 07:37:00.509974 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.509893    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5837851c-03f6-49c9-bce4-9ef14221d0a8-kube-api-access-9wxhw" (OuterVolumeSpecName: "kube-api-access-9wxhw") pod "5837851c-03f6-49c9-bce4-9ef14221d0a8" (UID: "5837851c-03f6-49c9-bce4-9ef14221d0a8"). InnerVolumeSpecName "kube-api-access-9wxhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 07:37:00.512652 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.512625    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0" (OuterVolumeSpecName: "model-storage") pod "5837851c-03f6-49c9-bce4-9ef14221d0a8" (UID: "5837851c-03f6-49c9-bce4-9ef14221d0a8"). InnerVolumeSpecName "pvc-bada7467-ebdd-48ed-8525-cac06e0bda69". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Jul 08 07:37:00.609246 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.609207    2571 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") on node \"ip-10-0-129-54.ec2.internal\" "
Jul 08 07:37:00.609246 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.609240    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wxhw\" (UniqueName: \"kubernetes.io/projected/5837851c-03f6-49c9-bce4-9ef14221d0a8-kube-api-access-9wxhw\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 07:37:00.727253 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.727229    2571 operation_generator.go:895] UnmountDevice succeeded for volume "pvc-bada7467-ebdd-48ed-8525-cac06e0bda69" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0") on node "ip-10-0-129-54.ec2.internal"
Jul 08 07:37:00.811447 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:00.811416    2571 reconciler_common.go:299] "Volume detached for volume \"pvc-bada7467-ebdd-48ed-8525-cac06e0bda69\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-03a3b4847c5ea08e0\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"csi-12508cac709fb80c00b6b231494ac5ab89dc7e7d0c604f2c45f16c875ec54220\""
Jul 08 07:37:01.307330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:01.307241    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4" event={"ID":"5837851c-03f6-49c9-bce4-9ef14221d0a8","Type":"ContainerDied","Data":"9e977e81b76681d074de74ac75e13adf122717d20013281c16433a0097edf846"}
Jul 08 07:37:01.307330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:01.307270    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"
Jul 08 07:37:01.307330 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:37:01.307281    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e977e81b76681d074de74ac75e13adf122717d20013281c16433a0097edf846"
Jul 08 07:38:22.226449 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.226364    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"]
Jul 08 07:38:22.227014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.226861    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5837851c-03f6-49c9-bce4-9ef14221d0a8" containerName="storage-initializer"
Jul 08 07:38:22.227014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.226880    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5837851c-03f6-49c9-bce4-9ef14221d0a8" containerName="storage-initializer"
Jul 08 07:38:22.227014 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.226967    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5837851c-03f6-49c9-bce4-9ef14221d0a8" containerName="storage-initializer"
Jul 08 07:38:22.229929 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.229909    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.232900 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.232879    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-epp-sa-dockercfg-pqfb5\""
Jul 08 07:38:22.233827 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.233806    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Jul 08 07:38:22.240678 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.240656    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"]
Jul 08 07:38:22.285465 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.285429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9deabdb-8a2c-4081-9b23-e151d0275be9-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.285641 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.285471    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9652l\" (UniqueName: \"kubernetes.io/projected/c9deabdb-8a2c-4081-9b23-e151d0275be9-kube-api-access-9652l\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.285641 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.285505    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9deabdb-8a2c-4081-9b23-e151d0275be9-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.285641 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.285582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c9deabdb-8a2c-4081-9b23-e151d0275be9-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.386628 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.386589    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9deabdb-8a2c-4081-9b23-e151d0275be9-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.386628 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.386629    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9652l\" (UniqueName: \"kubernetes.io/projected/c9deabdb-8a2c-4081-9b23-e151d0275be9-kube-api-access-9652l\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.386928 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.386650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9deabdb-8a2c-4081-9b23-e151d0275be9-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.386928 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.386701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c9deabdb-8a2c-4081-9b23-e151d0275be9-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.387138 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.387115    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9deabdb-8a2c-4081-9b23-e151d0275be9-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.387208 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.387129    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c9deabdb-8a2c-4081-9b23-e151d0275be9-tokenizer-uds\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.389306 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.389280    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9deabdb-8a2c-4081-9b23-e151d0275be9-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.395608 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.395584    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9652l\" (UniqueName: \"kubernetes.io/projected/c9deabdb-8a2c-4081-9b23-e151d0275be9-kube-api-access-9652l\") pod \"custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc\" (UID: \"c9deabdb-8a2c-4081-9b23-e151d0275be9\") " pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.542213 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.542127    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:22.686192 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.686162    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"]
Jul 08 07:38:22.687373 ip-10-0-129-54 kubenswrapper[2571]: W0708 07:38:22.687347    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9deabdb_8a2c_4081_9b23_e151d0275be9.slice/crio-b873c3aa0dbbfc96d2f4456c47247b0c9fcb34fce7173f7669a8858391b02714 WatchSource:0}: Error finding container b873c3aa0dbbfc96d2f4456c47247b0c9fcb34fce7173f7669a8858391b02714: Status 404 returned error can't find the container with id b873c3aa0dbbfc96d2f4456c47247b0c9fcb34fce7173f7669a8858391b02714
Jul 08 07:38:22.720671 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:22.720642    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc" event={"ID":"c9deabdb-8a2c-4081-9b23-e151d0275be9","Type":"ContainerStarted","Data":"b873c3aa0dbbfc96d2f4456c47247b0c9fcb34fce7173f7669a8858391b02714"}
Jul 08 07:38:23.726637 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:23.726607    2571 generic.go:356] "Generic (PLEG): container finished" podID="c9deabdb-8a2c-4081-9b23-e151d0275be9" containerID="44c7909793090dc28d9c83088a302634c290c1c411335157104ed52ef04710e4" exitCode=0
Jul 08 07:38:23.727048 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:23.726663    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc" event={"ID":"c9deabdb-8a2c-4081-9b23-e151d0275be9","Type":"ContainerDied","Data":"44c7909793090dc28d9c83088a302634c290c1c411335157104ed52ef04710e4"}
Jul 08 07:38:24.735072 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:24.735036    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc" event={"ID":"c9deabdb-8a2c-4081-9b23-e151d0275be9","Type":"ContainerStarted","Data":"54e4da05ac1f352a11802c23cecfe1cd956bd92421004e444cb17d187705dfb7"}
Jul 08 07:38:24.735459 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:24.735147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:38:24.755722 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:24.755653    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc" podStartSLOduration=2.004879973 podStartE2EDuration="2.75564029s" totalImagesPullingTime="0s" totalInitContainerRuntime="750.760317ms" isStatefulPod=false podCreationTimestamp="2026-07-08 07:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 07:38:24.754882379 +0000 UTC m=+2632.354172082" watchObservedRunningTime="2026-07-08 07:38:24.75564029 +0000 UTC m=+2632.354929993"
Jul 08 07:38:55.741275 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:38:55.741249    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc"
Jul 08 07:55:38.759651 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:55:38.759620    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 07:55:38.770772 ip-10-0-129-54 kubenswrapper[2571]: I0708 07:55:38.770750    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:09:34.954563 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:34.954494    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"]
Jul 08 08:09:34.958058 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:34.958039    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:34.960582 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:34.960557    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-vmg5h\""
Jul 08 08:09:34.960657 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:34.960611    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 08 08:09:34.969249 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:34.969231    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"]
Jul 08 08:09:35.030451 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.030416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k52v\" (UniqueName: \"kubernetes.io/projected/1b3129b5-df20-48e9-953b-ed36a68bdf87-kube-api-access-4k52v\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.030778 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.030750    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.031105 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.031083    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.031225 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.031166    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b3129b5-df20-48e9-953b-ed36a68bdf87-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.131942 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.131898    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.132152 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.131964    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.132152 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.132001    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b3129b5-df20-48e9-953b-ed36a68bdf87-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.132152 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.132035    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k52v\" (UniqueName: \"kubernetes.io/projected/1b3129b5-df20-48e9-953b-ed36a68bdf87-kube-api-access-4k52v\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.132440 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.132417    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.132645 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.132616    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.135224 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.135199    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b3129b5-df20-48e9-953b-ed36a68bdf87-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.143168 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.143128    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k52v\" (UniqueName: \"kubernetes.io/projected/1b3129b5-df20-48e9-953b-ed36a68bdf87-kube-api-access-4k52v\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.269660 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.269580    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:35.418777 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.418735    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"]
Jul 08 08:09:35.418903 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:09:35.418882    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3129b5_df20_48e9_953b_ed36a68bdf87.slice/crio-91b5d0b6ec99fe03646c1c61d447d84ce916a6c9df890cdd8a214b36566fd153 WatchSource:0}: Error finding container 91b5d0b6ec99fe03646c1c61d447d84ce916a6c9df890cdd8a214b36566fd153: Status 404 returned error can't find the container with id 91b5d0b6ec99fe03646c1c61d447d84ce916a6c9df890cdd8a214b36566fd153
Jul 08 08:09:35.424179 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:35.424161    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 08:09:36.365141 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:36.365103    2571 generic.go:356] "Generic (PLEG): container finished" podID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerID="71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db" exitCode=0
Jul 08 08:09:36.365539 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:36.365159    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" event={"ID":"1b3129b5-df20-48e9-953b-ed36a68bdf87","Type":"ContainerDied","Data":"71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db"}
Jul 08 08:09:36.365539 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:36.365207    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" event={"ID":"1b3129b5-df20-48e9-953b-ed36a68bdf87","Type":"ContainerStarted","Data":"91b5d0b6ec99fe03646c1c61d447d84ce916a6c9df890cdd8a214b36566fd153"}
Jul 08 08:09:37.373070 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:37.373040    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" event={"ID":"1b3129b5-df20-48e9-953b-ed36a68bdf87","Type":"ContainerStarted","Data":"06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"}
Jul 08 08:09:37.373460 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:37.373113    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:09:37.409705 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:09:37.409640    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" podStartSLOduration=2.652069744 podStartE2EDuration="3.409624939s" totalImagesPullingTime="0s" totalInitContainerRuntime="757.555195ms" isStatefulPod=false podCreationTimestamp="2026-07-08 08:09:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:09:37.405784764 +0000 UTC m=+4505.005074469" watchObservedRunningTime="2026-07-08 08:09:37.409624939 +0000 UTC m=+4505.008914643"
Jul 08 08:10:08.386744 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:10:08.386707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:11:13.752970 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.752898    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"]
Jul 08 08:11:13.756392 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.756376    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.759480 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.759459    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Jul 08 08:11:13.759606 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.759499    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-h4pmk\""
Jul 08 08:11:13.772958 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.772938    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"]
Jul 08 08:11:13.877482 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.877441    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.877701 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.877522    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.877701 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.877558    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8xhk\" (UniqueName: \"kubernetes.io/projected/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kube-api-access-q8xhk\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.877701 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.877655    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.978658 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.978618    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.978658 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.978653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8xhk\" (UniqueName: \"kubernetes.io/projected/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kube-api-access-q8xhk\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.978902 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.978756    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.978902 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.978846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.979124 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.979101    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kserve-provision-location\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.979193 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.979178    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tokenizer-uds\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.981591 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.981566    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:13.988793 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:13.988763    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8xhk\" (UniqueName: \"kubernetes.io/projected/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kube-api-access-q8xhk\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") " pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:14.066842 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:14.066763    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:14.226179 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:14.226148    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"]
Jul 08 08:11:14.227825 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:11:14.227797    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd70721_ac4f_4328_97ee_50ec6d538a3a.slice/crio-11f71d8f22c7be9e90abe46a2228686ae7b6e2c3d4a4888475826e08a70adaa9 WatchSource:0}: Error finding container 11f71d8f22c7be9e90abe46a2228686ae7b6e2c3d4a4888475826e08a70adaa9: Status 404 returned error can't find the container with id 11f71d8f22c7be9e90abe46a2228686ae7b6e2c3d4a4888475826e08a70adaa9
Jul 08 08:11:14.882586 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:14.882548    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" event={"ID":"5bd70721-ac4f-4328-97ee-50ec6d538a3a","Type":"ContainerStarted","Data":"29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45"}
Jul 08 08:11:14.883072 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:14.882607    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" event={"ID":"5bd70721-ac4f-4328-97ee-50ec6d538a3a","Type":"ContainerStarted","Data":"11f71d8f22c7be9e90abe46a2228686ae7b6e2c3d4a4888475826e08a70adaa9"}
Jul 08 08:11:15.888938 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:15.888897    2571 generic.go:356] "Generic (PLEG): container finished" podID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerID="29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45" exitCode=0
Jul 08 08:11:15.889510 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:15.888991    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" event={"ID":"5bd70721-ac4f-4328-97ee-50ec6d538a3a","Type":"ContainerDied","Data":"29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45"}
Jul 08 08:11:16.896724 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:16.896674    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" event={"ID":"5bd70721-ac4f-4328-97ee-50ec6d538a3a","Type":"ContainerStarted","Data":"45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"}
Jul 08 08:11:16.897117 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:16.896804    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:11:16.921771 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:16.921602    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" podStartSLOduration=3.148537357 podStartE2EDuration="3.921585952s" totalImagesPullingTime="0s" totalInitContainerRuntime="773.048595ms" isStatefulPod=false podCreationTimestamp="2026-07-08 08:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:11:16.920789262 +0000 UTC m=+4604.520078962" watchObservedRunningTime="2026-07-08 08:11:16.921585952 +0000 UTC m=+4604.520875655"
Jul 08 08:11:47.904326 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:11:47.904295    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:12:21.760300 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:21.760262    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"]
Jul 08 08:12:21.770757 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:12:21.770725    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:12:21.770757 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:12:21.770757 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:12:21.770757 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de" cmd=["/bin/sleep","15"]
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:21.770770    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" message="" err=<
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:12:21.770784    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="main" containerID="cri-o://06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"
Jul 08 08:12:21.771120 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:21.770820    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="main" containerID="cri-o://06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de" gracePeriod=60
Jul 08 08:12:22.036933 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.036909    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:12:22.139179 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139149    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-kserve-provision-location\") pod \"1b3129b5-df20-48e9-953b-ed36a68bdf87\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") "
Jul 08 08:12:22.139339 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139186    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b3129b5-df20-48e9-953b-ed36a68bdf87-tls-certs\") pod \"1b3129b5-df20-48e9-953b-ed36a68bdf87\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") "
Jul 08 08:12:22.139339 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139220    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k52v\" (UniqueName: \"kubernetes.io/projected/1b3129b5-df20-48e9-953b-ed36a68bdf87-kube-api-access-4k52v\") pod \"1b3129b5-df20-48e9-953b-ed36a68bdf87\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") "
Jul 08 08:12:22.139339 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139286    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-tokenizer-uds\") pod \"1b3129b5-df20-48e9-953b-ed36a68bdf87\" (UID: \"1b3129b5-df20-48e9-953b-ed36a68bdf87\") "
Jul 08 08:12:22.139622 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139592    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "1b3129b5-df20-48e9-953b-ed36a68bdf87" (UID: "1b3129b5-df20-48e9-953b-ed36a68bdf87"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:12:22.139762 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139744    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:12:22.140063 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.139979    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1b3129b5-df20-48e9-953b-ed36a68bdf87" (UID: "1b3129b5-df20-48e9-953b-ed36a68bdf87"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:12:22.141583 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.141558    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b3129b5-df20-48e9-953b-ed36a68bdf87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b3129b5-df20-48e9-953b-ed36a68bdf87" (UID: "1b3129b5-df20-48e9-953b-ed36a68bdf87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:12:22.141706 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.141583    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3129b5-df20-48e9-953b-ed36a68bdf87-kube-api-access-4k52v" (OuterVolumeSpecName: "kube-api-access-4k52v") pod "1b3129b5-df20-48e9-953b-ed36a68bdf87" (UID: "1b3129b5-df20-48e9-953b-ed36a68bdf87"). InnerVolumeSpecName "kube-api-access-4k52v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:12:22.240337 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.240307    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b3129b5-df20-48e9-953b-ed36a68bdf87-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:12:22.240337 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.240337    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b3129b5-df20-48e9-953b-ed36a68bdf87-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:12:22.240539 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.240347    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k52v\" (UniqueName: \"kubernetes.io/projected/1b3129b5-df20-48e9-953b-ed36a68bdf87-kube-api-access-4k52v\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:12:22.256036 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.255998    2571 generic.go:356] "Generic (PLEG): container finished" podID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerID="06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de" exitCode=0
Jul 08 08:12:22.256169 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.256054    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"
Jul 08 08:12:22.256169 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.256076    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" event={"ID":"1b3129b5-df20-48e9-953b-ed36a68bdf87","Type":"ContainerDied","Data":"06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"}
Jul 08 08:12:22.256169 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.256133    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf" event={"ID":"1b3129b5-df20-48e9-953b-ed36a68bdf87","Type":"ContainerDied","Data":"91b5d0b6ec99fe03646c1c61d447d84ce916a6c9df890cdd8a214b36566fd153"}
Jul 08 08:12:22.256169 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.256153    2571 scope.go:117] "RemoveContainer" containerID="06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"
Jul 08 08:12:22.266935 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.266914    2571 scope.go:117] "RemoveContainer" containerID="71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db"
Jul 08 08:12:22.275522 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.275502    2571 scope.go:117] "RemoveContainer" containerID="06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"
Jul 08 08:12:22.275824 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:12:22.275807    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de\": container with ID starting with 06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de not found: ID does not exist" containerID="06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"
Jul 08 08:12:22.275882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.275834    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de"} err="failed to get container status \"06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de\": rpc error: code = NotFound desc = could not find container \"06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de\": container with ID starting with 06a6c792d813b73b98e6b1e3195f4e94638311bfc4f62715a6c61824f5f0b0de not found: ID does not exist"
Jul 08 08:12:22.275882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.275851    2571 scope.go:117] "RemoveContainer" containerID="71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db"
Jul 08 08:12:22.276106 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:12:22.276091    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db\": container with ID starting with 71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db not found: ID does not exist" containerID="71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db"
Jul 08 08:12:22.276144 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.276110    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db"} err="failed to get container status \"71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db\": rpc error: code = NotFound desc = could not find container \"71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db\": container with ID starting with 71c43d633e0b3ff9ace5db35257fb95681b4d417d02d62ae97a77b73057360db not found: ID does not exist"
Jul 08 08:12:22.284517 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.284490    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"]
Jul 08 08:12:22.290105 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.290082    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-ldrsf"]
Jul 08 08:12:22.914627 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:22.914596    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" path="/var/lib/kubelet/pods/1b3129b5-df20-48e9-953b-ed36a68bdf87/volumes"
Jul 08 08:12:36.717658 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.717625    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"]
Jul 08 08:12:36.718076 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.718062    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="main"
Jul 08 08:12:36.718152 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.718079    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="main"
Jul 08 08:12:36.718152 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.718102    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="storage-initializer"
Jul 08 08:12:36.718152 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.718107    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="storage-initializer"
Jul 08 08:12:36.718251 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.718174    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b3129b5-df20-48e9-953b-ed36a68bdf87" containerName="main"
Jul 08 08:12:36.722585 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.722568    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.726635 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.726615    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-mgc45\""
Jul 08 08:12:36.726764 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.726617    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
Jul 08 08:12:36.737890 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.737863    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"]
Jul 08 08:12:36.782600 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.782573    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.782792 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.782610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.782792 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.782676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59731070-b1f6-49ee-a550-c43c0698d86b-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.782792 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.782744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xghvp\" (UniqueName: \"kubernetes.io/projected/59731070-b1f6-49ee-a550-c43c0698d86b-kube-api-access-xghvp\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.884057 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.884023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59731070-b1f6-49ee-a550-c43c0698d86b-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.884235 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.884073    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xghvp\" (UniqueName: \"kubernetes.io/projected/59731070-b1f6-49ee-a550-c43c0698d86b-kube-api-access-xghvp\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.884235 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.884117    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.884235 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.884139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.884547 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.884522    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.884716 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.884670    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.886619 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.886603    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59731070-b1f6-49ee-a550-c43c0698d86b-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:36.893798 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:36.893774    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xghvp\" (UniqueName: \"kubernetes.io/projected/59731070-b1f6-49ee-a550-c43c0698d86b-kube-api-access-xghvp\") pod \"stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:37.033276 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:37.033173    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:37.184430 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:37.184402    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"]
Jul 08 08:12:37.185367 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:12:37.185340    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59731070_b1f6_49ee_a550_c43c0698d86b.slice/crio-f77e87b1b991e8f9641fad23e9024860964c20d4cc3d9570582718a5a3b15336 WatchSource:0}: Error finding container f77e87b1b991e8f9641fad23e9024860964c20d4cc3d9570582718a5a3b15336: Status 404 returned error can't find the container with id f77e87b1b991e8f9641fad23e9024860964c20d4cc3d9570582718a5a3b15336
Jul 08 08:12:37.343299 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:37.343217    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" event={"ID":"59731070-b1f6-49ee-a550-c43c0698d86b","Type":"ContainerStarted","Data":"c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73"}
Jul 08 08:12:37.343299 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:37.343269    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" event={"ID":"59731070-b1f6-49ee-a550-c43c0698d86b","Type":"ContainerStarted","Data":"f77e87b1b991e8f9641fad23e9024860964c20d4cc3d9570582718a5a3b15336"}
Jul 08 08:12:38.350031 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:38.349999    2571 generic.go:356] "Generic (PLEG): container finished" podID="59731070-b1f6-49ee-a550-c43c0698d86b" containerID="c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73" exitCode=0
Jul 08 08:12:38.350399 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:38.350088    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" event={"ID":"59731070-b1f6-49ee-a550-c43c0698d86b","Type":"ContainerDied","Data":"c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73"}
Jul 08 08:12:39.357161 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:39.357123    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" event={"ID":"59731070-b1f6-49ee-a550-c43c0698d86b","Type":"ContainerStarted","Data":"3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"}
Jul 08 08:12:39.357620 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:39.357268    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:12:39.387631 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:12:39.387583    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" podStartSLOduration=2.49224626 podStartE2EDuration="3.387567196s" totalImagesPullingTime="0s" totalInitContainerRuntime="895.320936ms" isStatefulPod=false podCreationTimestamp="2026-07-08 08:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:12:39.387125387 +0000 UTC m=+4686.986415093" watchObservedRunningTime="2026-07-08 08:12:39.387567196 +0000 UTC m=+4686.986856900"
Jul 08 08:13:10.365332 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:13:10.365303    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:14:20.773278 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:20.773199    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"]
Jul 08 08:14:20.783585 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:14:20.783547    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:14:20.783585 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:14:20.783585 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:14:20.783585 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81" cmd=["/bin/sleep","15"]
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:20.783603    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" message="" err=<
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:14:20.783617    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="main" containerID="cri-o://45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"
Jul 08 08:14:20.783931 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:20.783656    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="main" containerID="cri-o://45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81" gracePeriod=60
Jul 08 08:14:21.040554 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.040528    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:14:21.085480 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.085007    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tokenizer-uds\") pod \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") "
Jul 08 08:14:21.085480 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.085092    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kserve-provision-location\") pod \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") "
Jul 08 08:14:21.085480 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.085151    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8xhk\" (UniqueName: \"kubernetes.io/projected/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kube-api-access-q8xhk\") pod \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") "
Jul 08 08:14:21.085480 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.085197    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tls-certs\") pod \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\" (UID: \"5bd70721-ac4f-4328-97ee-50ec6d538a3a\") "
Jul 08 08:14:21.086136 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.086090    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5bd70721-ac4f-4328-97ee-50ec6d538a3a" (UID: "5bd70721-ac4f-4328-97ee-50ec6d538a3a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:14:21.086494 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.086423    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5bd70721-ac4f-4328-97ee-50ec6d538a3a" (UID: "5bd70721-ac4f-4328-97ee-50ec6d538a3a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:14:21.088010 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.087978    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kube-api-access-q8xhk" (OuterVolumeSpecName: "kube-api-access-q8xhk") pod "5bd70721-ac4f-4328-97ee-50ec6d538a3a" (UID: "5bd70721-ac4f-4328-97ee-50ec6d538a3a"). InnerVolumeSpecName "kube-api-access-q8xhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:14:21.088182 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.088160    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5bd70721-ac4f-4328-97ee-50ec6d538a3a" (UID: "5bd70721-ac4f-4328-97ee-50ec6d538a3a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:14:21.186661 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.186632    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:14:21.186661 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.186659    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8xhk\" (UniqueName: \"kubernetes.io/projected/5bd70721-ac4f-4328-97ee-50ec6d538a3a-kube-api-access-q8xhk\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:14:21.186899 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.186668    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:14:21.186899 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.186719    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5bd70721-ac4f-4328-97ee-50ec6d538a3a-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:14:21.898496 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.898460    2571 generic.go:356] "Generic (PLEG): container finished" podID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerID="45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81" exitCode=0
Jul 08 08:14:21.898986 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.898511    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"
Jul 08 08:14:21.898986 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.898543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" event={"ID":"5bd70721-ac4f-4328-97ee-50ec6d538a3a","Type":"ContainerDied","Data":"45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"}
Jul 08 08:14:21.898986 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.898604    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k" event={"ID":"5bd70721-ac4f-4328-97ee-50ec6d538a3a","Type":"ContainerDied","Data":"11f71d8f22c7be9e90abe46a2228686ae7b6e2c3d4a4888475826e08a70adaa9"}
Jul 08 08:14:21.898986 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.898635    2571 scope.go:117] "RemoveContainer" containerID="45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"
Jul 08 08:14:21.908768 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.908752    2571 scope.go:117] "RemoveContainer" containerID="29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45"
Jul 08 08:14:21.917510 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.917491    2571 scope.go:117] "RemoveContainer" containerID="45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"
Jul 08 08:14:21.917789 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:14:21.917771    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81\": container with ID starting with 45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81 not found: ID does not exist" containerID="45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"
Jul 08 08:14:21.917834 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.917799    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81"} err="failed to get container status \"45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81\": rpc error: code = NotFound desc = could not find container \"45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81\": container with ID starting with 45875b8dbaa402e5a15949f3e5e1c919a667d5f5c4cd4a53c781dca970ee2d81 not found: ID does not exist"
Jul 08 08:14:21.917834 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.917827    2571 scope.go:117] "RemoveContainer" containerID="29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45"
Jul 08 08:14:21.918067 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:14:21.918046    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45\": container with ID starting with 29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45 not found: ID does not exist" containerID="29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45"
Jul 08 08:14:21.918103 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.918074    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45"} err="failed to get container status \"29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45\": rpc error: code = NotFound desc = could not find container \"29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45\": container with ID starting with 29cac0dfdbcc80fe255321cb9f5f43e41fed0bedd9ca9bc9379f01b200c27f45 not found: ID does not exist"
Jul 08 08:14:21.921288 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.921266    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"]
Jul 08 08:14:21.926022 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:21.926002    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schetml7k"]
Jul 08 08:14:22.914107 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:22.914075    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" path="/var/lib/kubelet/pods/5bd70721-ac4f-4328-97ee-50ec6d538a3a/volumes"
Jul 08 08:14:57.768661 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.768628    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"]
Jul 08 08:14:57.769140 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.769027    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="main"
Jul 08 08:14:57.769140 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.769043    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="main"
Jul 08 08:14:57.769140 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.769083    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="storage-initializer"
Jul 08 08:14:57.769140 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.769089    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="storage-initializer"
Jul 08 08:14:57.769271 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.769155    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd70721-ac4f-4328-97ee-50ec6d538a3a" containerName="main"
Jul 08 08:14:57.772249 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.772232    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.775447 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.775428    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-zczvq\""
Jul 08 08:14:57.775591 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.775571    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Jul 08 08:14:57.787072 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.787047    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"]
Jul 08 08:14:57.853944 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.853904    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq56b\" (UniqueName: \"kubernetes.io/projected/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kube-api-access-nq56b\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.853944 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.853948    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.854169 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.854037    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.854169 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.854103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.955106 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.955068    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.955301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.955142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq56b\" (UniqueName: \"kubernetes.io/projected/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kube-api-access-nq56b\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.955301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.955164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.955301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.955213    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.955615 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.955597    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.955678 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.955615    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tokenizer-uds\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.958082 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.958056    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:57.964807 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:57.964786    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq56b\" (UniqueName: \"kubernetes.io/projected/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kube-api-access-nq56b\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") " pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:58.082957 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:58.082868    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:14:58.228320 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:58.228189    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"]
Jul 08 08:14:58.231466 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:14:58.231437    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63189d2a_01d6_485d_b2a9_09d7bd48ad83.slice/crio-fbf9a386822557793b15dd0361ad3afdbd437726dee0b492fcb5dfefd73626e2 WatchSource:0}: Error finding container fbf9a386822557793b15dd0361ad3afdbd437726dee0b492fcb5dfefd73626e2: Status 404 returned error can't find the container with id fbf9a386822557793b15dd0361ad3afdbd437726dee0b492fcb5dfefd73626e2
Jul 08 08:14:59.098282 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:59.098251    2571 generic.go:356] "Generic (PLEG): container finished" podID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerID="308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1" exitCode=0
Jul 08 08:14:59.098664 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:59.098297    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" event={"ID":"63189d2a-01d6-485d-b2a9-09d7bd48ad83","Type":"ContainerDied","Data":"308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1"}
Jul 08 08:14:59.098664 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:14:59.098331    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" event={"ID":"63189d2a-01d6-485d-b2a9-09d7bd48ad83","Type":"ContainerStarted","Data":"fbf9a386822557793b15dd0361ad3afdbd437726dee0b492fcb5dfefd73626e2"}
Jul 08 08:15:00.105991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:00.105959    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" event={"ID":"63189d2a-01d6-485d-b2a9-09d7bd48ad83","Type":"ContainerStarted","Data":"e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"}
Jul 08 08:15:00.106414 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:00.106065    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:15:00.137368 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:00.137315    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" podStartSLOduration=2.391358162 podStartE2EDuration="3.137299416s" totalImagesPullingTime="0s" totalInitContainerRuntime="745.941254ms" isStatefulPod=false podCreationTimestamp="2026-07-08 08:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:15:00.130774962 +0000 UTC m=+4827.730064685" watchObservedRunningTime="2026-07-08 08:15:00.137299416 +0000 UTC m=+4827.736589118"
Jul 08 08:15:27.607885 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:27.607808    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"]
Jul 08 08:15:27.618173 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:15:27.618138    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:15:27.618173 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:15:27.618173 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:15:27.618173 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600" cmd=["/bin/sleep","15"]
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:27.618194    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" message="" err=<
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:15:27.618211    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="main" containerID="cri-o://3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"
Jul 08 08:15:27.618491 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:27.618268    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="main" containerID="cri-o://3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600" gracePeriod=60
Jul 08 08:15:27.883314 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:27.883290    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:15:28.059085 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059046    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-kserve-provision-location\") pod \"59731070-b1f6-49ee-a550-c43c0698d86b\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") "
Jul 08 08:15:28.059285 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059141    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-tokenizer-uds\") pod \"59731070-b1f6-49ee-a550-c43c0698d86b\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") "
Jul 08 08:15:28.059285 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059251    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xghvp\" (UniqueName: \"kubernetes.io/projected/59731070-b1f6-49ee-a550-c43c0698d86b-kube-api-access-xghvp\") pod \"59731070-b1f6-49ee-a550-c43c0698d86b\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") "
Jul 08 08:15:28.059285 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059279    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59731070-b1f6-49ee-a550-c43c0698d86b-tls-certs\") pod \"59731070-b1f6-49ee-a550-c43c0698d86b\" (UID: \"59731070-b1f6-49ee-a550-c43c0698d86b\") "
Jul 08 08:15:28.059450 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059381    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "59731070-b1f6-49ee-a550-c43c0698d86b" (UID: "59731070-b1f6-49ee-a550-c43c0698d86b"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:15:28.059709 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059672    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:15:28.059859 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.059795    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "59731070-b1f6-49ee-a550-c43c0698d86b" (UID: "59731070-b1f6-49ee-a550-c43c0698d86b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:15:28.061638 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.061609    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59731070-b1f6-49ee-a550-c43c0698d86b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59731070-b1f6-49ee-a550-c43c0698d86b" (UID: "59731070-b1f6-49ee-a550-c43c0698d86b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:15:28.061638 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.061621    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59731070-b1f6-49ee-a550-c43c0698d86b-kube-api-access-xghvp" (OuterVolumeSpecName: "kube-api-access-xghvp") pod "59731070-b1f6-49ee-a550-c43c0698d86b" (UID: "59731070-b1f6-49ee-a550-c43c0698d86b"). InnerVolumeSpecName "kube-api-access-xghvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:15:28.161211 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.161129    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xghvp\" (UniqueName: \"kubernetes.io/projected/59731070-b1f6-49ee-a550-c43c0698d86b-kube-api-access-xghvp\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:15:28.161211 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.161160    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59731070-b1f6-49ee-a550-c43c0698d86b-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:15:28.161211 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.161172    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59731070-b1f6-49ee-a550-c43c0698d86b-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:15:28.261066 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.261033    2571 generic.go:356] "Generic (PLEG): container finished" podID="59731070-b1f6-49ee-a550-c43c0698d86b" containerID="3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600" exitCode=0
Jul 08 08:15:28.261241 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.261088    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"
Jul 08 08:15:28.261241 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.261082    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" event={"ID":"59731070-b1f6-49ee-a550-c43c0698d86b","Type":"ContainerDied","Data":"3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"}
Jul 08 08:15:28.261241 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.261200    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf" event={"ID":"59731070-b1f6-49ee-a550-c43c0698d86b","Type":"ContainerDied","Data":"f77e87b1b991e8f9641fad23e9024860964c20d4cc3d9570582718a5a3b15336"}
Jul 08 08:15:28.261241 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.261227    2571 scope.go:117] "RemoveContainer" containerID="3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"
Jul 08 08:15:28.271436 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.271415    2571 scope.go:117] "RemoveContainer" containerID="c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73"
Jul 08 08:15:28.280660 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.280621    2571 scope.go:117] "RemoveContainer" containerID="3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"
Jul 08 08:15:28.281329 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:15:28.281155    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600\": container with ID starting with 3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600 not found: ID does not exist" containerID="3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"
Jul 08 08:15:28.281329 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.281188    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600"} err="failed to get container status \"3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600\": rpc error: code = NotFound desc = could not find container \"3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600\": container with ID starting with 3996da24fe490af7946c31f21fb77a4fe53973f81690abb2e1bff2c32b869600 not found: ID does not exist"
Jul 08 08:15:28.281329 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.281214    2571 scope.go:117] "RemoveContainer" containerID="c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73"
Jul 08 08:15:28.281734 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:15:28.281620    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73\": container with ID starting with c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73 not found: ID does not exist" containerID="c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73"
Jul 08 08:15:28.281734 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.281666    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73"} err="failed to get container status \"c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73\": rpc error: code = NotFound desc = could not find container \"c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73\": container with ID starting with c7202f261f19ea4d78ceb0cff81f3ce15642d9d5730f668cecc6427dca1a4f73 not found: ID does not exist"
Jul 08 08:15:28.284181 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.284160    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"]
Jul 08 08:15:28.287605 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.287578    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-754d57d4d6-rkgjf"]
Jul 08 08:15:28.913795 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:28.913757    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" path="/var/lib/kubelet/pods/59731070-b1f6-49ee-a550-c43c0698d86b/volumes"
Jul 08 08:15:31.113861 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:15:31.113832    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:18:07.202060 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.202025    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"]
Jul 08 08:18:07.214730 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:18:07.214660    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:18:07.214730 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:18:07.214730 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:18:07.214730 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5" cmd=["/bin/sleep","15"]
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.214739    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" message="" err=<
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:18:07.214752    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="main" containerID="cri-o://e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"
Jul 08 08:18:07.214984 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.214808    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="main" containerID="cri-o://e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5" gracePeriod=60
Jul 08 08:18:07.484430 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.484401    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:18:07.510926 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.510901    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tokenizer-uds\") pod \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") "
Jul 08 08:18:07.511118 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.510942    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kserve-provision-location\") pod \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") "
Jul 08 08:18:07.511118 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.511017    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tls-certs\") pod \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") "
Jul 08 08:18:07.511118 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.511064    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq56b\" (UniqueName: \"kubernetes.io/projected/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kube-api-access-nq56b\") pod \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\" (UID: \"63189d2a-01d6-485d-b2a9-09d7bd48ad83\") "
Jul 08 08:18:07.511299 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.511172    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "63189d2a-01d6-485d-b2a9-09d7bd48ad83" (UID: "63189d2a-01d6-485d-b2a9-09d7bd48ad83"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:18:07.511396 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.511372    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:18:07.511766 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.511740    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "63189d2a-01d6-485d-b2a9-09d7bd48ad83" (UID: "63189d2a-01d6-485d-b2a9-09d7bd48ad83"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:18:07.513451 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.513426    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kube-api-access-nq56b" (OuterVolumeSpecName: "kube-api-access-nq56b") pod "63189d2a-01d6-485d-b2a9-09d7bd48ad83" (UID: "63189d2a-01d6-485d-b2a9-09d7bd48ad83"). InnerVolumeSpecName "kube-api-access-nq56b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:18:07.513586 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.513569    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63189d2a-01d6-485d-b2a9-09d7bd48ad83" (UID: "63189d2a-01d6-485d-b2a9-09d7bd48ad83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:18:07.612334 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.612303    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63189d2a-01d6-485d-b2a9-09d7bd48ad83-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:18:07.612334 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.612333    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq56b\" (UniqueName: \"kubernetes.io/projected/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kube-api-access-nq56b\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:18:07.612334 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:07.612342    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63189d2a-01d6-485d-b2a9-09d7bd48ad83-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:18:08.114668 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.114635    2571 generic.go:356] "Generic (PLEG): container finished" podID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerID="e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5" exitCode=0
Jul 08 08:18:08.114873 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.114720    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"
Jul 08 08:18:08.114873 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.114718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" event={"ID":"63189d2a-01d6-485d-b2a9-09d7bd48ad83","Type":"ContainerDied","Data":"e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"}
Jul 08 08:18:08.114873 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.114837    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs" event={"ID":"63189d2a-01d6-485d-b2a9-09d7bd48ad83","Type":"ContainerDied","Data":"fbf9a386822557793b15dd0361ad3afdbd437726dee0b492fcb5dfefd73626e2"}
Jul 08 08:18:08.114873 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.114857    2571 scope.go:117] "RemoveContainer" containerID="e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"
Jul 08 08:18:08.124541 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.124490    2571 scope.go:117] "RemoveContainer" containerID="308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1"
Jul 08 08:18:08.133771 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.133752    2571 scope.go:117] "RemoveContainer" containerID="e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"
Jul 08 08:18:08.134022 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:18:08.134002    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5\": container with ID starting with e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5 not found: ID does not exist" containerID="e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"
Jul 08 08:18:08.134104 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.134033    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5"} err="failed to get container status \"e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5\": rpc error: code = NotFound desc = could not find container \"e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5\": container with ID starting with e2b8585d616ac7173955ce88abf12033f5622dfcae2ed78c874d6ccd9e8a9ce5 not found: ID does not exist"
Jul 08 08:18:08.134104 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.134080    2571 scope.go:117] "RemoveContainer" containerID="308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1"
Jul 08 08:18:08.134334 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:18:08.134316    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1\": container with ID starting with 308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1 not found: ID does not exist" containerID="308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1"
Jul 08 08:18:08.134372 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.134340    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1"} err="failed to get container status \"308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1\": rpc error: code = NotFound desc = could not find container \"308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1\": container with ID starting with 308065be8daa1431c490870aed9b79647b3ebc2dae43c862a7309d27e42072a1 not found: ID does not exist"
Jul 08 08:18:08.142100 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.142070    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"]
Jul 08 08:18:08.149028 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.148992    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-786c7dxnfs"]
Jul 08 08:18:08.915005 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:08.914973    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" path="/var/lib/kubelet/pods/63189d2a-01d6-485d-b2a9-09d7bd48ad83/volumes"
Jul 08 08:18:28.037257 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037224    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"]
Jul 08 08:18:28.037796 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037777    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="storage-initializer"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037799    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="storage-initializer"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037815    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="storage-initializer"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037824    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="storage-initializer"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037838    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="main"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037846    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="main"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037871    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="main"
Jul 08 08:18:28.037882 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.037879    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="main"
Jul 08 08:18:28.038229 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.038022    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="63189d2a-01d6-485d-b2a9-09d7bd48ad83" containerName="main"
Jul 08 08:18:28.038229 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.038039    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="59731070-b1f6-49ee-a550-c43c0698d86b" containerName="main"
Jul 08 08:18:28.041139 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.041119    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.044927 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.044909    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-zfwm8\""
Jul 08 08:18:28.059280 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.059254    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"]
Jul 08 08:18:28.112445 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112415    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-credential-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112618 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-data\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112618 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-workload-certs\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112618 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-workload-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112618 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-envoy\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112806 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112716    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f9c188-9368-4945-8007-0d0d53a746ca-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112806 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-token\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112806 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvvdh\" (UniqueName: \"kubernetes.io/projected/a3f9c188-9368-4945-8007-0d0d53a746ca-kube-api-access-gvvdh\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.112920 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.112806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-podinfo\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213283 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213254    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-workload-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213283 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-envoy\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f9c188-9368-4945-8007-0d0d53a746ca-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-token\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213376    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvvdh\" (UniqueName: \"kubernetes.io/projected/a3f9c188-9368-4945-8007-0d0d53a746ca-kube-api-access-gvvdh\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213396    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-podinfo\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213418    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-credential-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213440    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-data\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.213505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213467    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-workload-certs\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.214006 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213757    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-workload-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.214006 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213872    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-workload-certs\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.214006 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.213900    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-credential-socket\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.214186 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.214156    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f9c188-9368-4945-8007-0d0d53a746ca-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.214337 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.214313    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-data\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.215901 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.215879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-podinfo\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.215979 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.215876    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-envoy\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.223013 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.222991    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvvdh\" (UniqueName: \"kubernetes.io/projected/a3f9c188-9368-4945-8007-0d0d53a746ca-kube-api-access-gvvdh\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.224757 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.224735    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f9c188-9368-4945-8007-0d0d53a746ca-istio-token\") pod \"router-gateway-2-openshift-default-54c789bdc6-4nrs7\" (UID: \"a3f9c188-9368-4945-8007-0d0d53a746ca\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.351580 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.351539    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:28.503945 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.503915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"]
Jul 08 08:18:28.504734 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:18:28.504708    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f9c188_9368_4945_8007_0d0d53a746ca.slice/crio-fd6be745ce4595dcf0b83e21b4aeb27a79904c2798e30cfe8c0fba075a7f144c WatchSource:0}: Error finding container fd6be745ce4595dcf0b83e21b4aeb27a79904c2798e30cfe8c0fba075a7f144c: Status 404 returned error can't find the container with id fd6be745ce4595dcf0b83e21b4aeb27a79904c2798e30cfe8c0fba075a7f144c
Jul 08 08:18:28.506992 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.506962    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 08:18:28.507052 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.507025    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 08:18:28.507097 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:28.507053    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 08:18:29.225142 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:29.225112    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" event={"ID":"a3f9c188-9368-4945-8007-0d0d53a746ca","Type":"ContainerStarted","Data":"258c3f3419e58f72a472547464ab7fcdd48555f2993ad11ee2d2ccc5e28790dc"}
Jul 08 08:18:29.225503 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:29.225164    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" event={"ID":"a3f9c188-9368-4945-8007-0d0d53a746ca","Type":"ContainerStarted","Data":"fd6be745ce4595dcf0b83e21b4aeb27a79904c2798e30cfe8c0fba075a7f144c"}
Jul 08 08:18:29.248273 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:29.248226    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podStartSLOduration=1.2482108 podStartE2EDuration="1.2482108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:18:29.247624723 +0000 UTC m=+5036.846914450" watchObservedRunningTime="2026-07-08 08:18:29.2482108 +0000 UTC m=+5036.847500503"
Jul 08 08:18:29.352515 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:29.352481    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:29.358207 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:29.358179    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:30.230613 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:30.230579    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:30.231738 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:30.231716    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:43.306367 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:43.306340    2571 generic.go:356] "Generic (PLEG): container finished" podID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerID="258c3f3419e58f72a472547464ab7fcdd48555f2993ad11ee2d2ccc5e28790dc" exitCode=0
Jul 08 08:18:43.306867 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:43.306425    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" event={"ID":"a3f9c188-9368-4945-8007-0d0d53a746ca","Type":"ContainerDied","Data":"258c3f3419e58f72a472547464ab7fcdd48555f2993ad11ee2d2ccc5e28790dc"}
Jul 08 08:18:43.306971 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:43.306953    2571 scope.go:117] "RemoveContainer" containerID="258c3f3419e58f72a472547464ab7fcdd48555f2993ad11ee2d2ccc5e28790dc"
Jul 08 08:18:43.308282 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:43.308249    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 08:18:43.308360 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:43.308338    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 08:18:43.308401 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:43.308385    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 08 08:18:44.314519 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:44.314482    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" event={"ID":"a3f9c188-9368-4945-8007-0d0d53a746ca","Type":"ContainerStarted","Data":"ed722f7ac0b80046146f7d8c2a2540fa828c5c1d74ef87ac3755ae96d566809d"}
Jul 08 08:18:44.352165 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:44.352121    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:44.353762 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:44.353729    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podUID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 08 08:18:45.230889 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:45.230854    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:45.352910 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:45.352860    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podUID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 08 08:18:46.352261 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:46.352205    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podUID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 08 08:18:47.352468 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:47.352417    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podUID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 08 08:18:48.352970 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:48.352922    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podUID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 08 08:18:49.352240 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:49.352198    2571 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7" podUID="a3f9c188-9368-4945-8007-0d0d53a746ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 08 08:18:50.356847 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:50.356812    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:18:50.357909 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:18:50.357893    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-4nrs7"
Jul 08 08:19:03.218146 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.218116    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"]
Jul 08 08:19:03.221574 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.221557    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.227313 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.227290    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-epp-sa-dockercfg-9tlf6\""
Jul 08 08:19:03.227469 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.227298    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Jul 08 08:19:03.246903 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.246868    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"]
Jul 08 08:19:03.366620 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.366581    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.366822 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.366638    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.366822 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.366758    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.366900 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.366850    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzhz2\" (UniqueName: \"kubernetes.io/projected/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-kube-api-access-lzhz2\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.468417 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.468325    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzhz2\" (UniqueName: \"kubernetes.io/projected/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-kube-api-access-lzhz2\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.468417 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.468378    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.468417 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.468412    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.468663 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.468443    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.468812 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.468792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.468865 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.468827    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-tokenizer-uds\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.471206 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.471178    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.482872 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.482841    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzhz2\" (UniqueName: \"kubernetes.io/projected/63405bd3-69a5-4e13-bdfd-f5f34ab6f635-kube-api-access-lzhz2\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh\" (UID: \"63405bd3-69a5-4e13-bdfd-f5f34ab6f635\") " pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.531909 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.531867    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:03.693510 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:03.693484    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"]
Jul 08 08:19:03.694834 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:19:03.694806    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63405bd3_69a5_4e13_bdfd_f5f34ab6f635.slice/crio-8f1c033f38f38cec0beb50c1bd4a4f3c9710cc0b75d6ebf0a15c5cbfc3f3fad8 WatchSource:0}: Error finding container 8f1c033f38f38cec0beb50c1bd4a4f3c9710cc0b75d6ebf0a15c5cbfc3f3fad8: Status 404 returned error can't find the container with id 8f1c033f38f38cec0beb50c1bd4a4f3c9710cc0b75d6ebf0a15c5cbfc3f3fad8
Jul 08 08:19:04.425972 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:04.425924    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh" event={"ID":"63405bd3-69a5-4e13-bdfd-f5f34ab6f635","Type":"ContainerStarted","Data":"fb4ec50659ff64095b0c78c7a47163703c70ff163441c31d27a4e1943210b009"}
Jul 08 08:19:04.426358 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:04.425990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh" event={"ID":"63405bd3-69a5-4e13-bdfd-f5f34ab6f635","Type":"ContainerStarted","Data":"8f1c033f38f38cec0beb50c1bd4a4f3c9710cc0b75d6ebf0a15c5cbfc3f3fad8"}
Jul 08 08:19:05.432015 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:05.431978    2571 generic.go:356] "Generic (PLEG): container finished" podID="63405bd3-69a5-4e13-bdfd-f5f34ab6f635" containerID="fb4ec50659ff64095b0c78c7a47163703c70ff163441c31d27a4e1943210b009" exitCode=0
Jul 08 08:19:05.432538 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:05.432066    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh" event={"ID":"63405bd3-69a5-4e13-bdfd-f5f34ab6f635","Type":"ContainerDied","Data":"fb4ec50659ff64095b0c78c7a47163703c70ff163441c31d27a4e1943210b009"}
Jul 08 08:19:06.439772 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:06.439740    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh" event={"ID":"63405bd3-69a5-4e13-bdfd-f5f34ab6f635","Type":"ContainerStarted","Data":"dbb7295d2e8563de5b7aabb0f2b63ad4a274856db4dc7d111cbbe5b002b26841"}
Jul 08 08:19:06.440178 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:06.439835    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:19:06.467350 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:06.467299    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh" podStartSLOduration=2.6788888220000002 podStartE2EDuration="3.467281203s" totalImagesPullingTime="0s" totalInitContainerRuntime="788.392381ms" isStatefulPod=false podCreationTimestamp="2026-07-08 08:19:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:19:06.467039239 +0000 UTC m=+5074.066329158" watchObservedRunningTime="2026-07-08 08:19:06.467281203 +0000 UTC m=+5074.066570907"
Jul 08 08:19:37.448106 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:19:37.448073    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh"
Jul 08 08:23:20.113789 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.113656    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"]
Jul 08 08:23:20.118168 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.118147    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.120563 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.120542    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"tls-verification-test-epp-sa-dockercfg-psg9d\""
Jul 08 08:23:20.121283 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.121263    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"tls-verification-test-kserve-self-signed-certs\""
Jul 08 08:23:20.133602 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.133577    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"]
Jul 08 08:23:20.233109 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.233073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tokenizer-uds\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.233304 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.233150    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.233304 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.233280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74lbr\" (UniqueName: \"kubernetes.io/projected/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kube-api-access-74lbr\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.233400 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.233310    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kserve-provision-location\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.334063 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.334026    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74lbr\" (UniqueName: \"kubernetes.io/projected/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kube-api-access-74lbr\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.334063 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.334063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kserve-provision-location\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.334330 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.334104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tokenizer-uds\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.334330 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.334148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.334581 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.334553    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tokenizer-uds\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.334717 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.334597    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kserve-provision-location\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.336851 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.336830    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.342190 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.342167    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74lbr\" (UniqueName: \"kubernetes.io/projected/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kube-api-access-74lbr\") pod \"tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") " pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.433823 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.433732    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:20.581422 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.581389    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"]
Jul 08 08:23:20.583622 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:23:20.583592    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda713ebc9_158c_4e1f_9fee_6ef17e31ba75.slice/crio-933817fccca4d1e109ca801d0996d3d31f618e198d161ec6aaf1ce842ab88f4f WatchSource:0}: Error finding container 933817fccca4d1e109ca801d0996d3d31f618e198d161ec6aaf1ce842ab88f4f: Status 404 returned error can't find the container with id 933817fccca4d1e109ca801d0996d3d31f618e198d161ec6aaf1ce842ab88f4f
Jul 08 08:23:20.785311 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.785230    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" event={"ID":"a713ebc9-158c-4e1f-9fee-6ef17e31ba75","Type":"ContainerStarted","Data":"8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f"}
Jul 08 08:23:20.785311 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:20.785284    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" event={"ID":"a713ebc9-158c-4e1f-9fee-6ef17e31ba75","Type":"ContainerStarted","Data":"933817fccca4d1e109ca801d0996d3d31f618e198d161ec6aaf1ce842ab88f4f"}
Jul 08 08:23:21.792556 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:21.792514    2571 generic.go:356] "Generic (PLEG): container finished" podID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerID="8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f" exitCode=0
Jul 08 08:23:21.793020 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:21.792589    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" event={"ID":"a713ebc9-158c-4e1f-9fee-6ef17e31ba75","Type":"ContainerDied","Data":"8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f"}
Jul 08 08:23:22.799832 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:22.799796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" event={"ID":"a713ebc9-158c-4e1f-9fee-6ef17e31ba75","Type":"ContainerStarted","Data":"9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"}
Jul 08 08:23:22.800204 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:22.799900    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:53.807499 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:53.807469    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:23:53.829572 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:23:53.829515    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" podStartSLOduration=32.948428939 podStartE2EDuration="33.829494087s" totalImagesPullingTime="0s" totalInitContainerRuntime="881.065148ms" isStatefulPod=false podCreationTimestamp="2026-07-08 08:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:23:22.823590655 +0000 UTC m=+5330.422880360" watchObservedRunningTime="2026-07-08 08:23:53.829494087 +0000 UTC m=+5361.428783793"
Jul 08 08:25:35.692218 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:35.692186    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"]
Jul 08 08:25:35.703033 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:25:35.702998    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:25:35.703033 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:25:35.703033 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:25:35.703033 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46" cmd=["/bin/sleep","15"]
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:35.703053    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" message="" err=<
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:25:35.703068    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="main" containerID="cri-o://9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"
Jul 08 08:25:35.703344 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:35.703121    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="main" containerID="cri-o://9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46" gracePeriod=60
Jul 08 08:25:35.961269 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:35.961247    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:25:36.077166 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.077127    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74lbr\" (UniqueName: \"kubernetes.io/projected/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kube-api-access-74lbr\") pod \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") "
Jul 08 08:25:36.077166 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.077171    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tls-certs\") pod \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") "
Jul 08 08:25:36.077444 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.077208    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tokenizer-uds\") pod \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") "
Jul 08 08:25:36.077444 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.077300    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kserve-provision-location\") pod \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\" (UID: \"a713ebc9-158c-4e1f-9fee-6ef17e31ba75\") "
Jul 08 08:25:36.077587 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.077561    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "a713ebc9-158c-4e1f-9fee-6ef17e31ba75" (UID: "a713ebc9-158c-4e1f-9fee-6ef17e31ba75"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:25:36.077811 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.077790    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:25:36.078349 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.078319    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a713ebc9-158c-4e1f-9fee-6ef17e31ba75" (UID: "a713ebc9-158c-4e1f-9fee-6ef17e31ba75"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:25:36.079646 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.079615    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kube-api-access-74lbr" (OuterVolumeSpecName: "kube-api-access-74lbr") pod "a713ebc9-158c-4e1f-9fee-6ef17e31ba75" (UID: "a713ebc9-158c-4e1f-9fee-6ef17e31ba75"). InnerVolumeSpecName "kube-api-access-74lbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:25:36.079646 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.079625    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a713ebc9-158c-4e1f-9fee-6ef17e31ba75" (UID: "a713ebc9-158c-4e1f-9fee-6ef17e31ba75"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:25:36.178506 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.178476    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:25:36.178506 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.178504    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74lbr\" (UniqueName: \"kubernetes.io/projected/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-kube-api-access-74lbr\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:25:36.178506 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.178514    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a713ebc9-158c-4e1f-9fee-6ef17e31ba75-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:25:36.524114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.524079    2571 generic.go:356] "Generic (PLEG): container finished" podID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerID="9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46" exitCode=0
Jul 08 08:25:36.524295 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.524130    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"
Jul 08 08:25:36.524295 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.524166    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" event={"ID":"a713ebc9-158c-4e1f-9fee-6ef17e31ba75","Type":"ContainerDied","Data":"9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"}
Jul 08 08:25:36.524295 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.524220    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj" event={"ID":"a713ebc9-158c-4e1f-9fee-6ef17e31ba75","Type":"ContainerDied","Data":"933817fccca4d1e109ca801d0996d3d31f618e198d161ec6aaf1ce842ab88f4f"}
Jul 08 08:25:36.524295 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.524243    2571 scope.go:117] "RemoveContainer" containerID="9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"
Jul 08 08:25:36.536289 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.536269    2571 scope.go:117] "RemoveContainer" containerID="8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f"
Jul 08 08:25:36.544700 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.544667    2571 scope.go:117] "RemoveContainer" containerID="9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"
Jul 08 08:25:36.544969 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:25:36.544951    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46\": container with ID starting with 9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46 not found: ID does not exist" containerID="9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"
Jul 08 08:25:36.545036 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.544977    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46"} err="failed to get container status \"9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46\": rpc error: code = NotFound desc = could not find container \"9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46\": container with ID starting with 9d54cd498a5d55c5236fa07372601c1b06607d384b7e1d0e41af12fcc84adb46 not found: ID does not exist"
Jul 08 08:25:36.545036 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.544995    2571 scope.go:117] "RemoveContainer" containerID="8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f"
Jul 08 08:25:36.545268 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:25:36.545249    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f\": container with ID starting with 8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f not found: ID does not exist" containerID="8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f"
Jul 08 08:25:36.545337 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.545274    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f"} err="failed to get container status \"8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f\": rpc error: code = NotFound desc = could not find container \"8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f\": container with ID starting with 8fd3f980a18dc9046e0cce863e29b862f84af31a10de4b6192d43499e8b1701f not found: ID does not exist"
Jul 08 08:25:36.548677 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.548657    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"]
Jul 08 08:25:36.552827 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.552808    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-6f85476866-4xdmj"]
Jul 08 08:25:36.915252 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:25:36.915220    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" path="/var/lib/kubelet/pods/a713ebc9-158c-4e1f-9fee-6ef17e31ba75/volumes"
Jul 08 08:26:07.803639 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.803562    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"]
Jul 08 08:26:07.804114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.804001    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="storage-initializer"
Jul 08 08:26:07.804114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.804030    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="storage-initializer"
Jul 08 08:26:07.804114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.804046    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="main"
Jul 08 08:26:07.804114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.804051    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="main"
Jul 08 08:26:07.804114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.804111    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a713ebc9-158c-4e1f-9fee-6ef17e31ba75" containerName="main"
Jul 08 08:26:07.807446 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.807425    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.809972 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.809948    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"prestop-hook-test-epp-sa-dockercfg-fsd8d\""
Jul 08 08:26:07.810730 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.810712    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"prestop-hook-test-kserve-self-signed-certs\""
Jul 08 08:26:07.824032 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.824002    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"]
Jul 08 08:26:07.896862 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.896828    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-kserve-provision-location\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.897028 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.896908    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74mf\" (UniqueName: \"kubernetes.io/projected/1edc1518-20c7-4814-bf53-7954ded1a0fc-kube-api-access-n74mf\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.897028 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.896957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-tokenizer-uds\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.897028 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.896992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1edc1518-20c7-4814-bf53-7954ded1a0fc-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.997681 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.997641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-kserve-provision-location\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.997891 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.997757    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74mf\" (UniqueName: \"kubernetes.io/projected/1edc1518-20c7-4814-bf53-7954ded1a0fc-kube-api-access-n74mf\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.997891 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.997815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-tokenizer-uds\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.997891 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.997875    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1edc1518-20c7-4814-bf53-7954ded1a0fc-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.998188 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.998160    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-kserve-provision-location\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:07.998253 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:07.998205    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-tokenizer-uds\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:08.000412 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:08.000395    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1edc1518-20c7-4814-bf53-7954ded1a0fc-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:08.006958 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:08.006937    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74mf\" (UniqueName: \"kubernetes.io/projected/1edc1518-20c7-4814-bf53-7954ded1a0fc-kube-api-access-n74mf\") pod \"prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") " pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:08.117678 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:08.117649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:08.269717 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:08.269673    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"]
Jul 08 08:26:08.271929 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:26:08.271900    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edc1518_20c7_4814_bf53_7954ded1a0fc.slice/crio-deab1b844e9fd667f048682c45c750ddcdec3549ba015a9789ea4b779453f671 WatchSource:0}: Error finding container deab1b844e9fd667f048682c45c750ddcdec3549ba015a9789ea4b779453f671: Status 404 returned error can't find the container with id deab1b844e9fd667f048682c45c750ddcdec3549ba015a9789ea4b779453f671
Jul 08 08:26:08.693945 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:08.693907    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" event={"ID":"1edc1518-20c7-4814-bf53-7954ded1a0fc","Type":"ContainerStarted","Data":"4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161"}
Jul 08 08:26:08.694194 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:08.693956    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" event={"ID":"1edc1518-20c7-4814-bf53-7954ded1a0fc","Type":"ContainerStarted","Data":"deab1b844e9fd667f048682c45c750ddcdec3549ba015a9789ea4b779453f671"}
Jul 08 08:26:09.700895 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:09.700813    2571 generic.go:356] "Generic (PLEG): container finished" podID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerID="4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161" exitCode=0
Jul 08 08:26:09.700895 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:09.700883    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" event={"ID":"1edc1518-20c7-4814-bf53-7954ded1a0fc","Type":"ContainerDied","Data":"4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161"}
Jul 08 08:26:10.707473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:10.707434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" event={"ID":"1edc1518-20c7-4814-bf53-7954ded1a0fc","Type":"ContainerStarted","Data":"b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"}
Jul 08 08:26:10.707862 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:10.707545    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:26:10.730092 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:10.730030    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" podStartSLOduration=2.693744054 podStartE2EDuration="3.730010723s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.036266669s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:26:10.725536533 +0000 UTC m=+5498.324826238" watchObservedRunningTime="2026-07-08 08:26:10.730010723 +0000 UTC m=+5498.329300429"
Jul 08 08:26:41.714751 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:26:41.714724    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:28:40.493604 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.493516    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"]
Jul 08 08:28:40.506038 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:28:40.506004    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:28:40.506038 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:28:40.506038 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:28:40.506038 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40" cmd=["/bin/sleep","15"]
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.506051    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" message="" err=<
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:28:40.506062    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="main" containerID="cri-o://b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"
Jul 08 08:28:40.506359 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.506109    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="main" containerID="cri-o://b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40" gracePeriod=60
Jul 08 08:28:40.780040 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.780017    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:28:40.935522 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.935491    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-tokenizer-uds\") pod \"1edc1518-20c7-4814-bf53-7954ded1a0fc\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") "
Jul 08 08:28:40.935719 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.935533    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n74mf\" (UniqueName: \"kubernetes.io/projected/1edc1518-20c7-4814-bf53-7954ded1a0fc-kube-api-access-n74mf\") pod \"1edc1518-20c7-4814-bf53-7954ded1a0fc\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") "
Jul 08 08:28:40.935719 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.935601    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-kserve-provision-location\") pod \"1edc1518-20c7-4814-bf53-7954ded1a0fc\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") "
Jul 08 08:28:40.935719 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.935618    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1edc1518-20c7-4814-bf53-7954ded1a0fc-tls-certs\") pod \"1edc1518-20c7-4814-bf53-7954ded1a0fc\" (UID: \"1edc1518-20c7-4814-bf53-7954ded1a0fc\") "
Jul 08 08:28:40.935899 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.935817    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "1edc1518-20c7-4814-bf53-7954ded1a0fc" (UID: "1edc1518-20c7-4814-bf53-7954ded1a0fc"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:28:40.936348 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.936324    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1edc1518-20c7-4814-bf53-7954ded1a0fc" (UID: "1edc1518-20c7-4814-bf53-7954ded1a0fc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:28:40.937874 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.937852    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1edc1518-20c7-4814-bf53-7954ded1a0fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1edc1518-20c7-4814-bf53-7954ded1a0fc" (UID: "1edc1518-20c7-4814-bf53-7954ded1a0fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:28:40.937974 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:40.937915    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edc1518-20c7-4814-bf53-7954ded1a0fc-kube-api-access-n74mf" (OuterVolumeSpecName: "kube-api-access-n74mf") pod "1edc1518-20c7-4814-bf53-7954ded1a0fc" (UID: "1edc1518-20c7-4814-bf53-7954ded1a0fc"). InnerVolumeSpecName "kube-api-access-n74mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:28:41.036635 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.036551    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:28:41.036635 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.036577    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n74mf\" (UniqueName: \"kubernetes.io/projected/1edc1518-20c7-4814-bf53-7954ded1a0fc-kube-api-access-n74mf\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:28:41.036635 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.036587    2571 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1edc1518-20c7-4814-bf53-7954ded1a0fc-kserve-provision-location\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:28:41.036635 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.036597    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1edc1518-20c7-4814-bf53-7954ded1a0fc-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:28:41.533749 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.533716    2571 generic.go:356] "Generic (PLEG): container finished" podID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerID="b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40" exitCode=0
Jul 08 08:28:41.534250 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.533804    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" event={"ID":"1edc1518-20c7-4814-bf53-7954ded1a0fc","Type":"ContainerDied","Data":"b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"}
Jul 08 08:28:41.534250 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.533837    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"
Jul 08 08:28:41.534250 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.533856    2571 scope.go:117] "RemoveContainer" containerID="b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"
Jul 08 08:28:41.534250 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.533846    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b" event={"ID":"1edc1518-20c7-4814-bf53-7954ded1a0fc","Type":"ContainerDied","Data":"deab1b844e9fd667f048682c45c750ddcdec3549ba015a9789ea4b779453f671"}
Jul 08 08:28:41.543548 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.543534    2571 scope.go:117] "RemoveContainer" containerID="4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161"
Jul 08 08:28:41.552231 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.552214    2571 scope.go:117] "RemoveContainer" containerID="b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"
Jul 08 08:28:41.552487 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:28:41.552469    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40\": container with ID starting with b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40 not found: ID does not exist" containerID="b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"
Jul 08 08:28:41.552539 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.552495    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40"} err="failed to get container status \"b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40\": rpc error: code = NotFound desc = could not find container \"b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40\": container with ID starting with b7f1146df2eb5365c47d4b38d683849bfc7185bc96d55a2371f1479de68f6e40 not found: ID does not exist"
Jul 08 08:28:41.552539 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.552513    2571 scope.go:117] "RemoveContainer" containerID="4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161"
Jul 08 08:28:41.552813 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:28:41.552789    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161\": container with ID starting with 4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161 not found: ID does not exist" containerID="4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161"
Jul 08 08:28:41.552907 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.552817    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161"} err="failed to get container status \"4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161\": rpc error: code = NotFound desc = could not find container \"4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161\": container with ID starting with 4c180826b97eb78f2954a47150dd46c3fab458b347eb95ea7070c826f0389161 not found: ID does not exist"
Jul 08 08:28:41.556267 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.556247    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"]
Jul 08 08:28:41.560292 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:41.560267    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-78799f54fd-9xt5b"]
Jul 08 08:28:42.914761 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:28:42.914732    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" path="/var/lib/kubelet/pods/1edc1518-20c7-4814-bf53-7954ded1a0fc/volumes"
Jul 08 08:29:51.910883 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.910850    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"]
Jul 08 08:29:51.911301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.911243    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="main"
Jul 08 08:29:51.911301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.911254    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="main"
Jul 08 08:29:51.911301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.911266    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="storage-initializer"
Jul 08 08:29:51.911301 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.911272    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="storage-initializer"
Jul 08 08:29:51.911441 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.911347    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edc1518-20c7-4814-bf53-7954ded1a0fc" containerName="main"
Jul 08 08:29:51.915133 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.915114    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:51.918961 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.918942    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 08 08:29:51.929562 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:51.929542    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"]
Jul 08 08:29:52.052293 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.052260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-model-cache\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.052472 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.052300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-dshm\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.052472 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.052346    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmxgg\" (UniqueName: \"kubernetes.io/projected/93d84678-2865-4592-a0f4-c219255d022e-kube-api-access-zmxgg\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.052472 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.052363    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-home\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.052472 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.052426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-tmp-dir\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.052472 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.052465    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d84678-2865-4592-a0f4-c219255d022e-tls-certs\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.153803 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.153774    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-model-cache\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.153991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.153816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-dshm\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.153991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.153862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmxgg\" (UniqueName: \"kubernetes.io/projected/93d84678-2865-4592-a0f4-c219255d022e-kube-api-access-zmxgg\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.153991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.153888    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-home\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.153991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.153925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-tmp-dir\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.153991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.153953    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d84678-2865-4592-a0f4-c219255d022e-tls-certs\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.154282 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.154259    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-model-cache\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.154382 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.154362    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-home\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.154447 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.154355    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-tmp-dir\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.156310 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.156290    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-dshm\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.156429 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.156413    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d84678-2865-4592-a0f4-c219255d022e-tls-certs\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.162559 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.162502    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmxgg\" (UniqueName: \"kubernetes.io/projected/93d84678-2865-4592-a0f4-c219255d022e-kube-api-access-zmxgg\") pod \"rolling-upgrade-test-kserve-54bc76f55b-c5xr4\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.226460 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.226426    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"]
Jul 08 08:29:52.226648 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.226460    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:29:52.230174 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.229952    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.232393 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.232369    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-epp-sa-dockercfg-868xq\""
Jul 08 08:29:52.239398 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.239343    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"]
Jul 08 08:29:52.356972 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.356939    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tokenizer-uds\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.357161 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.356993    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6wnt\" (UniqueName: \"kubernetes.io/projected/3ea323fc-552c-46ae-ab86-1342db1dc3d5-kube-api-access-w6wnt\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.357161 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.357098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.384921 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.384863    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"]
Jul 08 08:29:52.458603 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.458579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.458751 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.458660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tokenizer-uds\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.458751 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.458714    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6wnt\" (UniqueName: \"kubernetes.io/projected/3ea323fc-552c-46ae-ab86-1342db1dc3d5-kube-api-access-w6wnt\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.459061 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.459041    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tokenizer-uds\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.461236 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.461210    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.467203 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.467181    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6wnt\" (UniqueName: \"kubernetes.io/projected/3ea323fc-552c-46ae-ab86-1342db1dc3d5-kube-api-access-w6wnt\") pod \"rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.562080 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.562047    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.710173 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.710148    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"]
Jul 08 08:29:52.711884 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:29:52.711855    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea323fc_552c_46ae_ab86_1342db1dc3d5.slice/crio-def079a6d3d0e43a86b7b5237bd07cf788c5cfa30fceff2eb4559ca771d5d077 WatchSource:0}: Error finding container def079a6d3d0e43a86b7b5237bd07cf788c5cfa30fceff2eb4559ca771d5d077: Status 404 returned error can't find the container with id def079a6d3d0e43a86b7b5237bd07cf788c5cfa30fceff2eb4559ca771d5d077
Jul 08 08:29:52.917747 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.917713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" event={"ID":"3ea323fc-552c-46ae-ab86-1342db1dc3d5","Type":"ContainerStarted","Data":"233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"}
Jul 08 08:29:52.918181 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.917759    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" event={"ID":"3ea323fc-552c-46ae-ab86-1342db1dc3d5","Type":"ContainerStarted","Data":"def079a6d3d0e43a86b7b5237bd07cf788c5cfa30fceff2eb4559ca771d5d077"}
Jul 08 08:29:52.918181 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.917789    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:29:52.919294 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.919271    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4" event={"ID":"93d84678-2865-4592-a0f4-c219255d022e","Type":"ContainerStarted","Data":"ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b"}
Jul 08 08:29:52.919398 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.919309    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4" event={"ID":"93d84678-2865-4592-a0f4-c219255d022e","Type":"ContainerStarted","Data":"3bd0aca157398f9232eba7959d897294184d5e3ab206c80d9338ed0de296595e"}
Jul 08 08:29:52.942243 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.942189    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" podStartSLOduration=0.942140302 podStartE2EDuration="942.140302ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:29:52.939829589 +0000 UTC m=+5720.539119295" watchObservedRunningTime="2026-07-08 08:29:52.942140302 +0000 UTC m=+5720.541430005"
Jul 08 08:29:52.961927 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:52.961828    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4" podStartSLOduration=1.96181085 podStartE2EDuration="1.96181085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:29:52.958823397 +0000 UTC m=+5720.558113132" watchObservedRunningTime="2026-07-08 08:29:52.96181085 +0000 UTC m=+5720.561100746"
Jul 08 08:29:53.227637 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:29:53.227550    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:30:02.226915 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:02.226876    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:30:02.243111 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:02.243079    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:30:02.989729 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:02.989678    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:30:23.926824 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:23.926743    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:30:38.508715 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:38.508658    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"]
Jul 08 08:30:43.617499 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.617456    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"]
Jul 08 08:30:43.628225 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:30:43.628170    2571 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 08 08:30:43.628225 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:30:43.628225 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:30:43.628225 ip-10-0-129-54 kubenswrapper[2571]:  > containerID="233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e" cmd=["/bin/sleep","15"]
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.628234    2571 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" message="" err=<
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]:  >
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:30:43.628251    2571 kuberuntime_container.go:754] "PreStop hook failed" err=<
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]:         , stdout: , stderr: , exit code -1
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]:  > pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" podUID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" containerName="main" containerID="cri-o://233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"
Jul 08 08:30:43.628572 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.628326    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" podUID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" containerName="main" containerID="cri-o://233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e" gracePeriod=60
Jul 08 08:30:43.879524 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.879450    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:30:43.963641 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.963611    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6wnt\" (UniqueName: \"kubernetes.io/projected/3ea323fc-552c-46ae-ab86-1342db1dc3d5-kube-api-access-w6wnt\") pod \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") "
Jul 08 08:30:43.963810 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.963659    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tokenizer-uds\") pod \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") "
Jul 08 08:30:43.963810 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.963729    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tls-certs\") pod \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\" (UID: \"3ea323fc-552c-46ae-ab86-1342db1dc3d5\") "
Jul 08 08:30:43.963995 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.963963    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "3ea323fc-552c-46ae-ab86-1342db1dc3d5" (UID: "3ea323fc-552c-46ae-ab86-1342db1dc3d5"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:30:43.965909 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.965890    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ea323fc-552c-46ae-ab86-1342db1dc3d5" (UID: "3ea323fc-552c-46ae-ab86-1342db1dc3d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:30:43.966071 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:43.966047    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ea323fc-552c-46ae-ab86-1342db1dc3d5-kube-api-access-w6wnt" (OuterVolumeSpecName: "kube-api-access-w6wnt") pod "3ea323fc-552c-46ae-ab86-1342db1dc3d5" (UID: "3ea323fc-552c-46ae-ab86-1342db1dc3d5"). InnerVolumeSpecName "kube-api-access-w6wnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:30:44.065393 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.065358    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6wnt\" (UniqueName: \"kubernetes.io/projected/3ea323fc-552c-46ae-ab86-1342db1dc3d5-kube-api-access-w6wnt\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:44.065393 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.065384    2571 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tokenizer-uds\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:44.065393 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.065393    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ea323fc-552c-46ae-ab86-1342db1dc3d5-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:44.200646 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.200563    2571 generic.go:356] "Generic (PLEG): container finished" podID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" containerID="233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e" exitCode=0
Jul 08 08:30:44.200646 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.200607    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"
Jul 08 08:30:44.200852 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.200652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" event={"ID":"3ea323fc-552c-46ae-ab86-1342db1dc3d5","Type":"ContainerDied","Data":"233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"}
Jul 08 08:30:44.200852 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.200731    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp" event={"ID":"3ea323fc-552c-46ae-ab86-1342db1dc3d5","Type":"ContainerDied","Data":"def079a6d3d0e43a86b7b5237bd07cf788c5cfa30fceff2eb4559ca771d5d077"}
Jul 08 08:30:44.200852 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.200754    2571 scope.go:117] "RemoveContainer" containerID="233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"
Jul 08 08:30:44.211606 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.211586    2571 scope.go:117] "RemoveContainer" containerID="233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"
Jul 08 08:30:44.211856 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:30:44.211839    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e\": container with ID starting with 233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e not found: ID does not exist" containerID="233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"
Jul 08 08:30:44.211912 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.211864    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e"} err="failed to get container status \"233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e\": rpc error: code = NotFound desc = could not find container \"233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e\": container with ID starting with 233843fda0426655d10843e03ccce883ae7c9adaee3945221a93d7b06833ba1e not found: ID does not exist"
Jul 08 08:30:44.224767 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.224740    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"]
Jul 08 08:30:44.231675 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.231650    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-pfplp"]
Jul 08 08:30:44.913994 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:44.913961    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" path="/var/lib/kubelet/pods/3ea323fc-552c-46ae-ab86-1342db1dc3d5/volumes"
Jul 08 08:30:53.531578 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.531533    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4" podUID="93d84678-2865-4592-a0f4-c219255d022e" containerName="main" containerID="cri-o://ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b" gracePeriod=45
Jul 08 08:30:53.790085 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.790030    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:30:53.862636 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862603    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-model-cache\") pod \"93d84678-2865-4592-a0f4-c219255d022e\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") "
Jul 08 08:30:53.862840 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862653    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmxgg\" (UniqueName: \"kubernetes.io/projected/93d84678-2865-4592-a0f4-c219255d022e-kube-api-access-zmxgg\") pod \"93d84678-2865-4592-a0f4-c219255d022e\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") "
Jul 08 08:30:53.862840 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862676    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-dshm\") pod \"93d84678-2865-4592-a0f4-c219255d022e\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") "
Jul 08 08:30:53.862840 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862712    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-home\") pod \"93d84678-2865-4592-a0f4-c219255d022e\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") "
Jul 08 08:30:53.862840 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862770    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d84678-2865-4592-a0f4-c219255d022e-tls-certs\") pod \"93d84678-2865-4592-a0f4-c219255d022e\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") "
Jul 08 08:30:53.862840 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862807    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-tmp-dir\") pod \"93d84678-2865-4592-a0f4-c219255d022e\" (UID: \"93d84678-2865-4592-a0f4-c219255d022e\") "
Jul 08 08:30:53.863104 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.862965    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-model-cache" (OuterVolumeSpecName: "model-cache") pod "93d84678-2865-4592-a0f4-c219255d022e" (UID: "93d84678-2865-4592-a0f4-c219255d022e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:30:53.863160 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.863095    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-home" (OuterVolumeSpecName: "home") pod "93d84678-2865-4592-a0f4-c219255d022e" (UID: "93d84678-2865-4592-a0f4-c219255d022e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:30:53.863160 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.863114    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "93d84678-2865-4592-a0f4-c219255d022e" (UID: "93d84678-2865-4592-a0f4-c219255d022e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:30:53.863160 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.863130    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-model-cache\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:53.864951 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.864924    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-dshm" (OuterVolumeSpecName: "dshm") pod "93d84678-2865-4592-a0f4-c219255d022e" (UID: "93d84678-2865-4592-a0f4-c219255d022e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:30:53.865059 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.864955    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93d84678-2865-4592-a0f4-c219255d022e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "93d84678-2865-4592-a0f4-c219255d022e" (UID: "93d84678-2865-4592-a0f4-c219255d022e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 08:30:53.865059 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.865011    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93d84678-2865-4592-a0f4-c219255d022e-kube-api-access-zmxgg" (OuterVolumeSpecName: "kube-api-access-zmxgg") pod "93d84678-2865-4592-a0f4-c219255d022e" (UID: "93d84678-2865-4592-a0f4-c219255d022e"). InnerVolumeSpecName "kube-api-access-zmxgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:30:53.964467 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.964429    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d84678-2865-4592-a0f4-c219255d022e-tls-certs\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:53.964467 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.964459    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-tmp-dir\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:53.964467 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.964469    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmxgg\" (UniqueName: \"kubernetes.io/projected/93d84678-2865-4592-a0f4-c219255d022e-kube-api-access-zmxgg\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:53.964738 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.964478    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-dshm\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:53.964738 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:53.964485    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d84678-2865-4592-a0f4-c219255d022e-home\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:30:54.255845 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.255804    2571 generic.go:356] "Generic (PLEG): container finished" podID="93d84678-2865-4592-a0f4-c219255d022e" containerID="ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b" exitCode=0
Jul 08 08:30:54.256038 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.255870    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"
Jul 08 08:30:54.256038 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.255878    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4" event={"ID":"93d84678-2865-4592-a0f4-c219255d022e","Type":"ContainerDied","Data":"ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b"}
Jul 08 08:30:54.256038 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.255943    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4" event={"ID":"93d84678-2865-4592-a0f4-c219255d022e","Type":"ContainerDied","Data":"3bd0aca157398f9232eba7959d897294184d5e3ab206c80d9338ed0de296595e"}
Jul 08 08:30:54.256038 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.255969    2571 scope.go:117] "RemoveContainer" containerID="ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b"
Jul 08 08:30:54.269059 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.269039    2571 scope.go:117] "RemoveContainer" containerID="ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b"
Jul 08 08:30:54.269331 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:30:54.269305    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b\": container with ID starting with ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b not found: ID does not exist" containerID="ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b"
Jul 08 08:30:54.269385 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.269334    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b"} err="failed to get container status \"ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b\": rpc error: code = NotFound desc = could not find container \"ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b\": container with ID starting with ee65f536a166cf41eafcbd41898f513bbbca4e7bfe6cb47e2cd7898eb8d6595b not found: ID does not exist"
Jul 08 08:30:54.284577 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.284548    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"]
Jul 08 08:30:54.288658 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.288636    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-c5xr4"]
Jul 08 08:30:54.914105 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:30:54.914069    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93d84678-2865-4592-a0f4-c219255d022e" path="/var/lib/kubelet/pods/93d84678-2865-4592-a0f4-c219255d022e/volumes"
Jul 08 08:31:02.848675 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.848643    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"]
Jul 08 08:31:02.849069 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.849038    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" containerName="main"
Jul 08 08:31:02.849069 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.849050    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" containerName="main"
Jul 08 08:31:02.849069 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.849065    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d84678-2865-4592-a0f4-c219255d022e" containerName="main"
Jul 08 08:31:02.849069 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.849070    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d84678-2865-4592-a0f4-c219255d022e" containerName="main"
Jul 08 08:31:02.849200 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.849144    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d84678-2865-4592-a0f4-c219255d022e" containerName="main"
Jul 08 08:31:02.849200 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.849156    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ea323fc-552c-46ae-ab86-1342db1dc3d5" containerName="main"
Jul 08 08:31:02.852072 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.852055    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:02.854797 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.854779    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 08 08:31:02.856742 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.856722    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-ltcjx\""
Jul 08 08:31:02.870942 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.870921    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"]
Jul 08 08:31:02.960286 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.960244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eefa50ba-9445-4ab3-b38f-87a0bdc82fb8-cert\") pod \"llmisvc-controller-manager-75d56bb94b-mnc4d\" (UID: \"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8\") " pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:02.960496 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:02.960304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qm9q\" (UniqueName: \"kubernetes.io/projected/eefa50ba-9445-4ab3-b38f-87a0bdc82fb8-kube-api-access-8qm9q\") pod \"llmisvc-controller-manager-75d56bb94b-mnc4d\" (UID: \"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8\") " pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:03.061610 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.061576    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eefa50ba-9445-4ab3-b38f-87a0bdc82fb8-cert\") pod \"llmisvc-controller-manager-75d56bb94b-mnc4d\" (UID: \"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8\") " pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:03.061817 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.061633    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qm9q\" (UniqueName: \"kubernetes.io/projected/eefa50ba-9445-4ab3-b38f-87a0bdc82fb8-kube-api-access-8qm9q\") pod \"llmisvc-controller-manager-75d56bb94b-mnc4d\" (UID: \"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8\") " pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:03.064260 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.064236    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eefa50ba-9445-4ab3-b38f-87a0bdc82fb8-cert\") pod \"llmisvc-controller-manager-75d56bb94b-mnc4d\" (UID: \"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8\") " pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:03.071156 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.071124    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qm9q\" (UniqueName: \"kubernetes.io/projected/eefa50ba-9445-4ab3-b38f-87a0bdc82fb8-kube-api-access-8qm9q\") pod \"llmisvc-controller-manager-75d56bb94b-mnc4d\" (UID: \"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8\") " pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:03.162379 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.162298    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:03.317539 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.317514    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"]
Jul 08 08:31:03.319059 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:31:03.319027    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeefa50ba_9445_4ab3_b38f_87a0bdc82fb8.slice/crio-fa42d371958374a2e24cb4529b7d6dd53a1f23b641893b50026d9bd2fcba8474 WatchSource:0}: Error finding container fa42d371958374a2e24cb4529b7d6dd53a1f23b641893b50026d9bd2fcba8474: Status 404 returned error can't find the container with id fa42d371958374a2e24cb4529b7d6dd53a1f23b641893b50026d9bd2fcba8474
Jul 08 08:31:03.320434 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:03.320414    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 08:31:04.319115 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:04.319080    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d" event={"ID":"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8","Type":"ContainerStarted","Data":"fa42d371958374a2e24cb4529b7d6dd53a1f23b641893b50026d9bd2fcba8474"}
Jul 08 08:31:07.338829 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:07.338797    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d" event={"ID":"eefa50ba-9445-4ab3-b38f-87a0bdc82fb8","Type":"ContainerStarted","Data":"e62c2da29caa47f40202dd61183e4cc40676ce57f88831377b0aaf1185c2f87f"}
Jul 08 08:31:07.339267 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:07.338851    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:31:07.360248 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:07.360194    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d" podStartSLOduration=2.272485071 podStartE2EDuration="5.360174778s" totalImagesPullingTime="3.087689707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:31:07.35774795 +0000 UTC m=+5794.957037659" watchObservedRunningTime="2026-07-08 08:31:07.360174778 +0000 UTC m=+5794.959464485"
Jul 08 08:31:38.347212 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:31:38.347177    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-75d56bb94b-mnc4d"
Jul 08 08:33:33.428082 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:33.428013    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:33:33.439480 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:33.439445    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:33:37.523043 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.523000    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pmpmw/must-gather-59rmb"]
Jul 08 08:33:37.527088 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.527059    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.530112 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.530090    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pmpmw\"/\"kube-root-ca.crt\""
Jul 08 08:33:37.532593 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.532569    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pmpmw\"/\"openshift-service-ca.crt\""
Jul 08 08:33:37.561078 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.561046    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmpmw/must-gather-59rmb"]
Jul 08 08:33:37.619884 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.619853    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq8dl\" (UniqueName: \"kubernetes.io/projected/9e902b2b-2b32-45e2-a360-684d3b6530bf-kube-api-access-dq8dl\") pod \"must-gather-59rmb\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") " pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.620036 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.619891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e902b2b-2b32-45e2-a360-684d3b6530bf-must-gather-output\") pod \"must-gather-59rmb\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") " pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.721329 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.721294    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq8dl\" (UniqueName: \"kubernetes.io/projected/9e902b2b-2b32-45e2-a360-684d3b6530bf-kube-api-access-dq8dl\") pod \"must-gather-59rmb\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") " pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.721329 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.721332    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e902b2b-2b32-45e2-a360-684d3b6530bf-must-gather-output\") pod \"must-gather-59rmb\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") " pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.721713 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.721667    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e902b2b-2b32-45e2-a360-684d3b6530bf-must-gather-output\") pod \"must-gather-59rmb\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") " pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.741130 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.741095    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq8dl\" (UniqueName: \"kubernetes.io/projected/9e902b2b-2b32-45e2-a360-684d3b6530bf-kube-api-access-dq8dl\") pod \"must-gather-59rmb\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") " pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.837008 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.836931    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:33:37.990933 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:37.990903    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmpmw/must-gather-59rmb"]
Jul 08 08:33:37.993705 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:33:37.993659    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e902b2b_2b32_45e2_a360_684d3b6530bf.slice/crio-f8754f619fe38b4b49772657dea00cec5b341c4a8bf5f9aca4f1503c928a4110 WatchSource:0}: Error finding container f8754f619fe38b4b49772657dea00cec5b341c4a8bf5f9aca4f1503c928a4110: Status 404 returned error can't find the container with id f8754f619fe38b4b49772657dea00cec5b341c4a8bf5f9aca4f1503c928a4110
Jul 08 08:33:38.166911 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:38.166872    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmpmw/must-gather-59rmb" event={"ID":"9e902b2b-2b32-45e2-a360-684d3b6530bf","Type":"ContainerStarted","Data":"f8754f619fe38b4b49772657dea00cec5b341c4a8bf5f9aca4f1503c928a4110"}
Jul 08 08:33:45.219670 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:45.219581    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmpmw/must-gather-59rmb" event={"ID":"9e902b2b-2b32-45e2-a360-684d3b6530bf","Type":"ContainerStarted","Data":"a98e004bd484d5ce60ab2af92e8757ed9d4d9a5263fb677ea405e436a69240dc"}
Jul 08 08:33:45.219670 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:45.219623    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmpmw/must-gather-59rmb" event={"ID":"9e902b2b-2b32-45e2-a360-684d3b6530bf","Type":"ContainerStarted","Data":"fb1bb9b6301e2807637566b245072512e1c32555a89e59d9f6752b6ce7b2d173"}
Jul 08 08:33:45.240251 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:45.240203    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pmpmw/must-gather-59rmb" podStartSLOduration=1.54576336 podStartE2EDuration="8.240187089s" totalImagesPullingTime="6.694423729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:33:45.237714302 +0000 UTC m=+5952.837004008" watchObservedRunningTime="2026-07-08 08:33:45.240187089 +0000 UTC m=+5952.839476813"
Jul 08 08:33:55.442046 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.442016    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:33:55.468448 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.468415    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:33:55.485087 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.485062    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:33:55.720835 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.720745    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:33:55.737665 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.737641    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:33:55.844868 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.844844    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:33:55.859914 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:55.859891    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:33:56.587410 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:56.587385    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:33:56.588167 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:56.588133    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:33:56.600669 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:56.600643    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:33:56.601108 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:56.601090    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:33:56.692555 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:56.692530    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:33:56.703763 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:56.703741    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:33:58.794783 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:58.794759    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:33:58.807505 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:58.807482    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:33:58.830711 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:58.830659    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:33:59.054826 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.054739    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:33:59.066259 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.066238    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:33:59.170529 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.170503    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:33:59.182273 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.182250    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:33:59.940629 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.940597    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:33:59.941465 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.941446    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:33:59.962902 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.962873    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:33:59.963205 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:33:59.963188    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:00.055017 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:00.054990    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:00.066871 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:00.066845    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:02.092994 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.092963    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:02.107589 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.107563    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:02.123836 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.123813    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:02.345207 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.345116    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:02.361549 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.361522    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:02.477860 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.477821    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:02.495996 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:02.495968    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:03.244373 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:03.244345    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:03.245114 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:03.245087    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:03.260211 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:03.260182    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:03.260522 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:03.260496    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:03.352095 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:03.352067    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:03.364819 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:03.364793    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:05.379803 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.379777    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:05.391181 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.391159    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:05.407468 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.407444    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:05.658868 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.658782    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:05.671831 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.671805    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:05.788297 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.788269    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:05.799182 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:05.799153    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:06.609107 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:06.609075    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:06.610027 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:06.610006    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:06.625582 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:06.625557    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:06.626022 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:06.626007    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:06.727845 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:06.727815    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:06.740053 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:06.740024    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:08.752874 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:08.752843    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:08.763944 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:08.763920    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:08.777554 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:08.777532    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:09.027907 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.027819    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:09.037586 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.037563    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:09.170912 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.170887    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:09.182797 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.182764    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:09.875460 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.875426    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:09.876947 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.876925    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:09.890781 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.890759    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:09.891294 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.891254    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:09.981490 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.981467    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:09.992955 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:09.992931    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:12.032425 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.032395    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:12.049903 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.049878    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:12.067726 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.067702    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:12.293941 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.293877    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:12.313903 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.313876    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:12.450521 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.450493    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:12.471584 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:12.471550    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:13.221031 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:13.220999    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:13.222357 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:13.222335    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:13.235160 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:13.235141    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:13.235581 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:13.235568    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:13.323461 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:13.323435    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:13.334997 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:13.334969    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:15.380625 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.380597    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:15.393791 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.393764    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:15.409527 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.409504    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:15.630372 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.630344    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:15.645511 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.645449    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:15.762820 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.762792    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:15.778028 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:15.777989    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:16.585142 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:16.585116    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:16.585860 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:16.585814    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:16.598017 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:16.597998    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:16.598444 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:16.598432    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:16.692643 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:16.692616    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:16.713883 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:16.713857    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:18.728361 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:18.728329    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:18.740828 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:18.740808    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:18.755742 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:18.755716    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:18.971164 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:18.971138    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:18.983749 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:18.983660    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:19.112633 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.112561    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:19.123794 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.123765    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:19.871333 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.871309    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:19.872185 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.872154    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:19.886378 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.886334    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:19.886833 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.886811    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:19.979944 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.979919    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:19.991605 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:19.991569    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:22.033550 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.033518    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:22.053711 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.053668    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:22.071062 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.071040    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:22.311273 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.311184    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:22.323519 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.323486    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:22.443781 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.443757    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:22.456426 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:22.456384    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:23.222004 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:23.221977    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:23.222951 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:23.222933    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:23.237040 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:23.237016    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:23.237544 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:23.237528    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:23.333068 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:23.333041    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:23.349987 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:23.349967    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:25.276222 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.276191    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:25.288291 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.288250    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:25.306478 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.306446    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:25.538351 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.538263    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:25.550661 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.550636    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:25.666905 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.666868    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:25.679306 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:25.679282    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:26.392230 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:26.392194    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:26.393003 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:26.392977    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:26.405471 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:26.405445    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:26.405975 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:26.405952    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:26.499980 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:26.499950    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:26.510330 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:26.510308    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:28.460707 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.460662    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:28.474777 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.474735    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:28.489398 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.489352    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:28.721864 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.721772    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:28.733605 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.733573    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:28.849470 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.849443    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:28.860810 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:28.860781    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:29.550430 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:29.550399    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:29.551269 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:29.551240    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:29.564129 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:29.564106    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:29.564589 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:29.564547    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:29.654246 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:29.654210    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:29.666245 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:29.666207    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:31.591021 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.590984    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:31.602741 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.602671    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:31.617737 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.617704    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:31.867187 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.867152    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:31.879053 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.879017    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:31.979270 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.979241    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:31.992239 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:31.992208    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:32.678595 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:32.678561    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:32.679700 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:32.679620    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:32.692209 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:32.692188    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:32.692422 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:32.692405    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:32.776001 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:32.775972    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:32.789912 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:32.789856    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:34.813954 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:34.813918    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:34.824819 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:34.824793    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:34.839138 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:34.839117    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:35.059822 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.059797    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:35.072890 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.072806    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:35.179490 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.179465    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:35.196033 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.196009    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:35.915639 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.915608    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:35.916818 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.916796    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:35.929751 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.929728    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:35.930278 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:35.930244    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:36.028500 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:36.028466    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:36.042921 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:36.042882    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:38.051943 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.051914    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:38.070932 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.070904    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:38.089468 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.089442    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:38.323725 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.323641    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:38.341981 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.341956    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:38.520758 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.520724    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:38.539923 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:38.539896    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:39.286421 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:39.286393    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:39.287149 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:39.287131    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:39.301407 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:39.301371    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:39.301971 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:39.301936    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:39.390583 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:39.390558    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:39.402619 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:39.402581    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:41.419453 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.419404    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:41.436318 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.436255    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:41.456679 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.456637    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:41.722468 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.722393    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:41.746453 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.746429    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:41.877204 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.877171    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:41.897957 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:41.897928    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:42.691352 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:42.691323    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:42.692234 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:42.692213    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:42.706170 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:42.706139    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:42.706652 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:42.706635    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:42.808202 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:42.808174    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:42.820576 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:42.820554    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:44.766790 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:44.766748    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:44.779651 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:44.779624    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:44.793927 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:44.793907    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:45.016991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.016920    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:45.027887 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.027863    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:45.142545 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.142519    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:45.153719 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.153672    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:45.886669 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.886642    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:45.887485 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.887467    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:45.903023 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.902994    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:45.903449 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.903436    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:45.992639 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:45.992609    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:46.004933 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:46.004907    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:49.588926 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.588902    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:49.600868 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.600830    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:49.615931 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.615906    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:49.852540 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.852514    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:49.864071 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.864049    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:49.973027 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.973001    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:49.984332 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:49.984309    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:50.685295 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:50.685262    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:50.686381 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:50.686358    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:50.700046 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:50.700008    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:50.700561 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:50.700542    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:50.812421 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:50.812382    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:50.823026 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:50.823004    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:52.829774 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:52.829738    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:52.846233 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:52.846206    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:52.866131 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:52.866106    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:53.144090 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:53.144062    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:53.164078 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:53.164031    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:53.297494 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:53.297451    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:53.309993 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:53.309961    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:54.021009 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:54.020924    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:54.021864 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:54.021847    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:54.039246 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:54.039222    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:54.039809 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:54.039790    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:54.146306 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:54.146281    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:54.163266 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:54.163234    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:56.203829 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.203790    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:56.214995 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.214967    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:56.230177 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.230144    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:56.446396 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.446371    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:56.457715 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.457628    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:56.557745 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.557715    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:56.569597 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:56.569575    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:34:57.253076 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:57.253048    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:34:57.254015 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:57.253988    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:34:57.266831 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:57.266801    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:34:57.267206 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:57.267190    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:34:57.350944 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:57.350914    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:34:57.362424 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:57.362400    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:34:59.358799 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.358766    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:34:59.370401 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.370373    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:34:59.384670 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.384641    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:34:59.590460 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.590426    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:34:59.600559 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.600537    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:34:59.726075 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.725997    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:34:59.765659 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:34:59.765627    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:35:00.489768 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:00.489737    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:35:00.490421 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:00.490403    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:35:00.507819 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:00.507793    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:35:00.508406 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:00.508387    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:35:00.628380 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:00.628343    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:35:00.651386 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:00.651363    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:35:02.779700 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:02.779654    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:35:02.807108 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:02.807083    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:35:02.830055 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:02.829986    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:35:03.058573 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:03.058483    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:35:03.075305 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:03.075276    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:35:03.203009 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:03.202980    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:35:03.221581 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:03.221555    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:35:04.018453 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:04.018416    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:35:04.019035 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:04.019019    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:35:04.033990 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:04.033963    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:35:04.034491 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:04.034474    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:35:04.126372 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:04.126345    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:35:04.137740 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:04.137704    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:35:06.125073 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.125045    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/main/0.log"
Jul 08 08:35:06.139793 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.139768    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_custom-route-timeout-test-kserve-router-scheduler-65985dc4bddbc_c9deabdb-8a2c-4081-9b23-e151d0275be9/storage-initializer/0.log"
Jul 08 08:35:06.155906 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.155879    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_e2e-pvc-model-download-vfrk4_5837851c-03f6-49c9-bce4-9ef14221d0a8/storage-initializer/0.log"
Jul 08 08:35:06.385730 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.385645    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/main/0.log"
Jul 08 08:35:06.400458 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.400432    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schenqx6z_7b24e8e6-7f75-4a57-a998-e2c566c96672/storage-initializer/0.log"
Jul 08 08:35:06.502267 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.502236    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/main/0.log"
Jul 08 08:35:06.514281 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:06.514246    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-sche92hd4_685b6e6b-dedb-4dc0-ba22-2711b4b10f95/storage-initializer/0.log"
Jul 08 08:35:07.304771 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:07.304738    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/2.log"
Jul 08 08:35:07.305395 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:07.305380    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-799f46c59b-kwjm2_f353d789-6bd9-4653-a007-ac3dab8cbbe6/istio-proxy/3.log"
Jul 08 08:35:07.318091 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:07.318071    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/1.log"
Jul 08 08:35:07.318477 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:07.318454    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-54c789bdc6-4nrs7_a3f9c188-9368-4945-8007-0d0d53a746ca/istio-proxy/0.log"
Jul 08 08:35:07.408852 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:07.408824    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/main/0.log"
Jul 08 08:35:07.419784 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:07.419761    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-with-refs-pd-test-kserve-router-scheduler-6ddb8947brh5kh_63405bd3-69a5-4e13-bdfd-f5f34ab6f635/storage-initializer/0.log"
Jul 08 08:35:14.422550 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:14.422520    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:15.850019 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:15.849978    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:17.350328 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:17.350293    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:18.852589 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:18.852552    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:20.285130 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:20.285096    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:21.753959 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:21.753926    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:23.217261 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:23.217222    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:24.722052 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:24.722017    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:26.132662 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:26.132612    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:27.535726 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:27.535662    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:28.929296 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:28.929268    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:30.401974 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:30.401936    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:31.844208 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:31.844173    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:33.281315 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:33.281277    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:34.728465 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:34.728424    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:36.177870 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:36.177837    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:37.596073 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:37.596018    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:39.014322 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:39.014260    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:40.436745 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:40.436710    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:41.936672 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:41.936633    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:43.378447 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:43.378417    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:44.811774 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:44.811741    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:46.223829 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:46.223792    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:47.673917 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:47.673874    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:49.124936 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:49.124902    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:50.546496 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:50.546457    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:51.945780 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:51.945742    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:53.341025 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:53.340977    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:54.800325 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:54.800293    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:56.242738 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:56.242667    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:57.636751 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:57.636680    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:35:59.154643 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:35:59.154587    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:00.590359 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:00.590328    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:02.109758 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:02.109723    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:03.677420 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:03.677380    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:05.101501 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:05.101448    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:06.534358 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:06.534318    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:07.937755 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:07.937718    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:09.419003 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:09.418972    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:10.819549 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:10.819502    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:12.251390 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:12.251361    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:13.704613 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:13.704577    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:15.210986 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:15.210947    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:16.659287 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:16.659248    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:18.154450 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:18.154398    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:19.557389 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:19.557354    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:20.966462 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:20.966432    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:22.347618 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:22.347588    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:23.786288 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:23.786257    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:25.275325 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:25.275290    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:26.706489 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:26.706437    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:28.166206 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:28.166177    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:29.577263 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:29.577228    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:30.995090 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:30.995060    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:32.458523 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:32.458488    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:33.891465 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:33.891435    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:35.337571 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:35.337529    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:36.784587 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:36.784528    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:38.212844 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:38.212813    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:39.667346 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:39.667310    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:41.124386 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:41.124354    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:42.565508 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:42.565466    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:44.341320 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:44.341285    2571 generic.go:356] "Generic (PLEG): container finished" podID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerID="fb1bb9b6301e2807637566b245072512e1c32555a89e59d9f6752b6ce7b2d173" exitCode=0
Jul 08 08:36:44.341755 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:44.341347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmpmw/must-gather-59rmb" event={"ID":"9e902b2b-2b32-45e2-a360-684d3b6530bf","Type":"ContainerDied","Data":"fb1bb9b6301e2807637566b245072512e1c32555a89e59d9f6752b6ce7b2d173"}
Jul 08 08:36:44.341806 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:44.341753    2571 scope.go:117] "RemoveContainer" containerID="fb1bb9b6301e2807637566b245072512e1c32555a89e59d9f6752b6ce7b2d173"
Jul 08 08:36:44.397335 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:44.397309    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pmpmw_must-gather-59rmb_9e902b2b-2b32-45e2-a360-684d3b6530bf/gather/0.log"
Jul 08 08:36:45.335080 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.335049    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cc4px/must-gather-xvnkx"]
Jul 08 08:36:45.340085 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.340064    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.346125 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.346087    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cc4px\"/\"openshift-service-ca.crt\""
Jul 08 08:36:45.346473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.346178    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cc4px\"/\"kube-root-ca.crt\""
Jul 08 08:36:45.352676 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.352645    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cc4px/must-gather-xvnkx"]
Jul 08 08:36:45.437063 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.437033    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e-must-gather-output\") pod \"must-gather-xvnkx\" (UID: \"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e\") " pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.437238 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.437082    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f672m\" (UniqueName: \"kubernetes.io/projected/afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e-kube-api-access-f672m\") pod \"must-gather-xvnkx\" (UID: \"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e\") " pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.538408 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.538373    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e-must-gather-output\") pod \"must-gather-xvnkx\" (UID: \"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e\") " pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.538589 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.538440    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f672m\" (UniqueName: \"kubernetes.io/projected/afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e-kube-api-access-f672m\") pod \"must-gather-xvnkx\" (UID: \"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e\") " pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.538838 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.538820    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e-must-gather-output\") pod \"must-gather-xvnkx\" (UID: \"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e\") " pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.553062 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.553039    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f672m\" (UniqueName: \"kubernetes.io/projected/afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e-kube-api-access-f672m\") pod \"must-gather-xvnkx\" (UID: \"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e\") " pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.649812 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.649781    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cc4px/must-gather-xvnkx"
Jul 08 08:36:45.793523 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.793499    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cc4px/must-gather-xvnkx"]
Jul 08 08:36:45.795404 ip-10-0-129-54 kubenswrapper[2571]: W0708 08:36:45.795377    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafbfbcbd_d6b1_4d8e_ae28_47cf15a86a7e.slice/crio-4e8586d147f67b492a03e88ec9b840d52892284cc0e1917e30a7a207484bcf96 WatchSource:0}: Error finding container 4e8586d147f67b492a03e88ec9b840d52892284cc0e1917e30a7a207484bcf96: Status 404 returned error can't find the container with id 4e8586d147f67b492a03e88ec9b840d52892284cc0e1917e30a7a207484bcf96
Jul 08 08:36:45.797070 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:45.797051    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 08:36:46.355342 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:46.355307    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cc4px/must-gather-xvnkx" event={"ID":"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e","Type":"ContainerStarted","Data":"4e8586d147f67b492a03e88ec9b840d52892284cc0e1917e30a7a207484bcf96"}
Jul 08 08:36:47.366502 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:47.366469    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cc4px/must-gather-xvnkx" event={"ID":"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e","Type":"ContainerStarted","Data":"0ff2380d361ea0f7a4259e27a39dad50c3285e0eccea18ca78da6b5573e52c8e"}
Jul 08 08:36:47.366502 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:47.366512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cc4px/must-gather-xvnkx" event={"ID":"afbfbcbd-d6b1-4d8e-ae28-47cf15a86a7e","Type":"ContainerStarted","Data":"5eda6911ef1b895dcb31b45c4417f7d4f705319aa6adbb87129adc0c6c3dfc64"}
Jul 08 08:36:47.386132 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:47.384905    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cc4px/must-gather-xvnkx" podStartSLOduration=1.347596134 podStartE2EDuration="2.384883044s" totalImagesPullingTime="1.03728691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 08:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:36:47.383183895 +0000 UTC m=+6134.982473599" watchObservedRunningTime="2026-07-08 08:36:47.384883044 +0000 UTC m=+6134.984172748"
Jul 08 08:36:48.518466 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:48.518409    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6qqpr_486b3034-61ca-49ad-9061-bf7c9369447c/global-pull-secret-syncer/0.log"
Jul 08 08:36:48.705044 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:48.705001    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kd766_66b62ff9-9859-419a-9aa0-365b4f25229b/konnectivity-agent/0.log"
Jul 08 08:36:48.789130 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:48.789053    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-54.ec2.internal_c8ad56ce288e22a2bd7487c8f13b9062/haproxy/0.log"
Jul 08 08:36:50.141604 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.141567    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-pmpmw/must-gather-59rmb"]
Jul 08 08:36:50.142524 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.141806    2571 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-pmpmw/must-gather-59rmb" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="copy" containerID="cri-o://a98e004bd484d5ce60ab2af92e8757ed9d4d9a5263fb677ea405e436a69240dc" gracePeriod=2
Jul 08 08:36:50.146627 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:36:50.146576    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-59rmb\" is forbidden: User \"system:node:ip-10-0-129-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-pmpmw\": no relationship found between node 'ip-10-0-129-54.ec2.internal' and this object" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:36:50.147280 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.147254    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-pmpmw/must-gather-59rmb"]
Jul 08 08:36:50.404168 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.404027    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pmpmw_must-gather-59rmb_9e902b2b-2b32-45e2-a360-684d3b6530bf/copy/0.log"
Jul 08 08:36:50.407743 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.407562    2571 generic.go:356] "Generic (PLEG): container finished" podID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerID="a98e004bd484d5ce60ab2af92e8757ed9d4d9a5263fb677ea405e436a69240dc" exitCode=143
Jul 08 08:36:50.498842 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.498796    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pmpmw_must-gather-59rmb_9e902b2b-2b32-45e2-a360-684d3b6530bf/copy/0.log"
Jul 08 08:36:50.503705 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.502368    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:36:50.509562 ip-10-0-129-54 kubenswrapper[2571]: E0708 08:36:50.508015    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-59rmb\" is forbidden: User \"system:node:ip-10-0-129-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-pmpmw\": no relationship found between node 'ip-10-0-129-54.ec2.internal' and this object" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:36:50.593661 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.593619    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq8dl\" (UniqueName: \"kubernetes.io/projected/9e902b2b-2b32-45e2-a360-684d3b6530bf-kube-api-access-dq8dl\") pod \"9e902b2b-2b32-45e2-a360-684d3b6530bf\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") "
Jul 08 08:36:50.593865 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.593679    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e902b2b-2b32-45e2-a360-684d3b6530bf-must-gather-output\") pod \"9e902b2b-2b32-45e2-a360-684d3b6530bf\" (UID: \"9e902b2b-2b32-45e2-a360-684d3b6530bf\") "
Jul 08 08:36:50.615473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.615439    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e902b2b-2b32-45e2-a360-684d3b6530bf-kube-api-access-dq8dl" (OuterVolumeSpecName: "kube-api-access-dq8dl") pod "9e902b2b-2b32-45e2-a360-684d3b6530bf" (UID: "9e902b2b-2b32-45e2-a360-684d3b6530bf"). InnerVolumeSpecName "kube-api-access-dq8dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 08:36:50.620712 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.620137    2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e902b2b-2b32-45e2-a360-684d3b6530bf-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "9e902b2b-2b32-45e2-a360-684d3b6530bf" (UID: "9e902b2b-2b32-45e2-a360-684d3b6530bf"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 08:36:50.695186 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.695092    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq8dl\" (UniqueName: \"kubernetes.io/projected/9e902b2b-2b32-45e2-a360-684d3b6530bf-kube-api-access-dq8dl\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:36:50.695405 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.695389    2571 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e902b2b-2b32-45e2-a360-684d3b6530bf-must-gather-output\") on node \"ip-10-0-129-54.ec2.internal\" DevicePath \"\""
Jul 08 08:36:50.919603 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:50.919568    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" path="/var/lib/kubelet/pods/9e902b2b-2b32-45e2-a360-684d3b6530bf/volumes"
Jul 08 08:36:51.417925 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:51.417888    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pmpmw_must-gather-59rmb_9e902b2b-2b32-45e2-a360-684d3b6530bf/copy/0.log"
Jul 08 08:36:51.419137 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:51.418416    2571 scope.go:117] "RemoveContainer" containerID="a98e004bd484d5ce60ab2af92e8757ed9d4d9a5263fb677ea405e436a69240dc"
Jul 08 08:36:51.419137 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:51.418601    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmpmw/must-gather-59rmb"
Jul 08 08:36:51.434724 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:51.434348    2571 scope.go:117] "RemoveContainer" containerID="fb1bb9b6301e2807637566b245072512e1c32555a89e59d9f6752b6ce7b2d173"
Jul 08 08:36:55.379711 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:55.379553    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-cvspb_46ca0a88-03ba-4de1-a7e0-fdf989eccf31/manager/0.log"
Jul 08 08:36:56.692868 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.692793    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/alertmanager/0.log"
Jul 08 08:36:56.724703 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.724653    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/config-reloader/0.log"
Jul 08 08:36:56.749279 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.749252    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/kube-rbac-proxy-web/0.log"
Jul 08 08:36:56.774112 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.774058    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/kube-rbac-proxy/0.log"
Jul 08 08:36:56.824713 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.822528    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/kube-rbac-proxy-metric/0.log"
Jul 08 08:36:56.850166 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.849979    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/prom-label-proxy/0.log"
Jul 08 08:36:56.890874 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.890811    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_348be2f3-97e5-4013-b12b-5db73dff5b1a/init-config-reloader/0.log"
Jul 08 08:36:56.973259 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:56.972498    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-qvvg6_f5bec0a8-4daa-4dc8-ad04-9b399c5f5826/cluster-monitoring-operator/0.log"
Jul 08 08:36:57.141191 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.141156    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-744f84b76c-rj9cq_c037f44f-2206-465a-bfe4-5a0864515456/metrics-server/0.log"
Jul 08 08:36:57.447165 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.447129    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7tjbk_77423e1e-a673-4ff5-aa50-d220f54dab03/node-exporter/0.log"
Jul 08 08:36:57.478736 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.478706    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7tjbk_77423e1e-a673-4ff5-aa50-d220f54dab03/kube-rbac-proxy/0.log"
Jul 08 08:36:57.510031 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.509998    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7tjbk_77423e1e-a673-4ff5-aa50-d220f54dab03/init-textfile/0.log"
Jul 08 08:36:57.669564 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.669470    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/prometheus/0.log"
Jul 08 08:36:57.703833 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.703770    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/config-reloader/0.log"
Jul 08 08:36:57.733286 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.733246    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/thanos-sidecar/0.log"
Jul 08 08:36:57.761820 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.761791    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/kube-rbac-proxy-web/0.log"
Jul 08 08:36:57.789936 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.789908    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/kube-rbac-proxy/0.log"
Jul 08 08:36:57.823245 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.823210    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/kube-rbac-proxy-thanos/0.log"
Jul 08 08:36:57.854276 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.854248    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a5c269d5-c208-4018-862d-cf99e46dea23/init-config-reloader/0.log"
Jul 08 08:36:57.890206 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.890168    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7w546_a65b84cb-1739-4de2-aa09-a9b7e61e6705/prometheus-operator/0.log"
Jul 08 08:36:57.918529 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.918498    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7w546_a65b84cb-1739-4de2-aa09-a9b7e61e6705/kube-rbac-proxy/0.log"
Jul 08 08:36:57.944986 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.944904    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-n6hv5_662f75e8-d26c-4975-8e9e-ba12328a648b/prometheus-operator-admission-webhook/0.log"
Jul 08 08:36:57.983244 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:57.983212    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-847cf488d5-2pjrx_61878bb7-e686-4acb-b1f5-a9b0b6c686ff/telemeter-client/0.log"
Jul 08 08:36:58.018185 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:58.018160    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-847cf488d5-2pjrx_61878bb7-e686-4acb-b1f5-a9b0b6c686ff/reload/0.log"
Jul 08 08:36:58.051586 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:58.051559    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-847cf488d5-2pjrx_61878bb7-e686-4acb-b1f5-a9b0b6c686ff/kube-rbac-proxy/0.log"
Jul 08 08:36:59.416639 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:36:59.416608    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-qxs7d_ae13f9c0-58e6-44a3-a5ba-ddcd0174c5a8/networking-console-plugin/0.log"
Jul 08 08:37:00.145804 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.145773    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"]
Jul 08 08:37:00.146296 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.146272    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="copy"
Jul 08 08:37:00.146296 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.146296    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="copy"
Jul 08 08:37:00.146473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.146321    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="gather"
Jul 08 08:37:00.146473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.146331    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="gather"
Jul 08 08:37:00.146473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.146433    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="copy"
Jul 08 08:37:00.146473 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.146443    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e902b2b-2b32-45e2-a360-684d3b6530bf" containerName="gather"
Jul 08 08:37:00.154240 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.154205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.158546 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.158516    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cc4px\"/\"default-dockercfg-zzsrs\""
Jul 08 08:37:00.160262 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.160240    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"]
Jul 08 08:37:00.228249 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.228212    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-podres\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.228436 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.228272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-sys\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.228436 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.228390    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-proc\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.228549 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.228433    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-lib-modules\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.228549 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.228529    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4sqm\" (UniqueName: \"kubernetes.io/projected/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-kube-api-access-s4sqm\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.329878 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.329829    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-podres\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330057 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.329908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-sys\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330057 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.329978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-proc\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330057 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.330005    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-lib-modules\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330228 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.330055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4sqm\" (UniqueName: \"kubernetes.io/projected/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-kube-api-access-s4sqm\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330591 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.330557    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-podres\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330711 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.330643    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-sys\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330777 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.330715    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-proc\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.330829 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.330818    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-lib-modules\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.341933 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.341905    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4sqm\" (UniqueName: \"kubernetes.io/projected/0d6fa563-8e5c-4ca9-8ad0-31175c95d882-kube-api-access-s4sqm\") pod \"perf-node-gather-daemonset-c9dgb\" (UID: \"0d6fa563-8e5c-4ca9-8ad0-31175c95d882\") " pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.479881 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.479792    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:00.571991 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.571885    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-765c4ccff4-242ps_1c9142b1-8507-43d0-92a8-851fb514b3c4/console/0.log"
Jul 08 08:37:00.661935 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:00.661850    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"]
Jul 08 08:37:01.052550 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:01.052450    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"]
Jul 08 08:37:01.057998 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:01.057960    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/e2e-pvc-model-download-vfrk4"]
Jul 08 08:37:01.497942 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:01.497900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb" event={"ID":"0d6fa563-8e5c-4ca9-8ad0-31175c95d882","Type":"ContainerStarted","Data":"6c4c3f5218370dd4875c13ea99bd4bdc868f1f9c0ed34784393ed246b2904a0b"}
Jul 08 08:37:01.498422 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:01.497970    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb" event={"ID":"0d6fa563-8e5c-4ca9-8ad0-31175c95d882","Type":"ContainerStarted","Data":"d259e39abafc3b5fc78a536f65042b87341ecdac53b51294edd7333428b2450c"}
Jul 08 08:37:01.498422 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:01.498023    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:01.522337 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:01.522285    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb" podStartSLOduration=1.522265212 podStartE2EDuration="1.522265212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 08:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 08:37:01.520069893 +0000 UTC m=+6149.119359587" watchObservedRunningTime="2026-07-08 08:37:01.522265212 +0000 UTC m=+6149.121554914"
Jul 08 08:37:02.575829 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:02.575806    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g468f_e96bc4c3-a102-47da-8a20-fa8c17a05c83/dns/0.log"
Jul 08 08:37:02.612791 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:02.612764    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g468f_e96bc4c3-a102-47da-8a20-fa8c17a05c83/kube-rbac-proxy/0.log"
Jul 08 08:37:02.797068 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:02.797033    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b8bd9_e1fd811b-ecd4-4720-b927-556e91ace214/dns-node-resolver/0.log"
Jul 08 08:37:02.930093 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:02.930051    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5837851c-03f6-49c9-bce4-9ef14221d0a8" path="/var/lib/kubelet/pods/5837851c-03f6-49c9-bce4-9ef14221d0a8/volumes"
Jul 08 08:37:03.555472 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:03.555444    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kkdj4_65032d2e-e6b0-4469-9921-3273ef4e5f75/node-ca/0.log"
Jul 08 08:37:06.075073 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:06.075051    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fm69n_f46102e8-813f-45bd-bcad-cac9e30b7d70/serve-healthcheck-canary/0.log"
Jul 08 08:37:06.632237 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:06.632201    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-g2pqf_00afd1f2-a8a8-4192-861f-733bc074b996/insights-operator/0.log"
Jul 08 08:37:06.633085 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:06.633050    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-g2pqf_00afd1f2-a8a8-4192-861f-733bc074b996/insights-operator/1.log"
Jul 08 08:37:06.657875 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:06.657846    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-glfl6_9a81c997-4343-4bda-a495-4753336ff8bb/kube-rbac-proxy/0.log"
Jul 08 08:37:06.683257 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:06.683212    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-glfl6_9a81c997-4343-4bda-a495-4753336ff8bb/exporter/0.log"
Jul 08 08:37:06.710604 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:06.710559    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-glfl6_9a81c997-4343-4bda-a495-4753336ff8bb/extractor/0.log"
Jul 08 08:37:07.520119 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:07.520092    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cc4px/perf-node-gather-daemonset-c9dgb"
Jul 08 08:37:10.283581 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:10.283522    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58c59b5fc8-fbnwl_18d572c8-8155-4737-8260-cdf19c0ed818/manager/0.log"
Jul 08 08:37:10.360460 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:10.360426    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-p86rw_421a4d10-350e-494a-ab78-906c728552a8/openshift-lws-operator/0.log"
Jul 08 08:37:11.245792 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:11.245756    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-75d56bb94b-mnc4d_eefa50ba-9445-4ab3-b38f-87a0bdc82fb8/manager/0.log"
Jul 08 08:37:11.276314 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:11.276285    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-4tbxf_2a79acbc-c478-430d-9726-96d07eb6a90b/server/0.log"
Jul 08 08:37:12.131395 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:12.131355    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-9rsxr_c4a13e5c-3a04-43dd-95d4-bbe56c03898f/manager/0.log"
Jul 08 08:37:12.169338 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:12.169268    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-7t9ph_c24181de-68b3-464d-a375-783658f553eb/s3-init/0.log"
Jul 08 08:37:12.191397 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:12.191367    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-7t9ph_c24181de-68b3-464d-a375-783658f553eb/download-hf-model/0.log"
Jul 08 08:37:18.231570 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:18.231505    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hwn2m_09c5f560-1820-422f-823b-7c90342c0793/kube-storage-version-migrator-operator/1.log"
Jul 08 08:37:18.233087 ip-10-0-129-54 kubenswrapper[2571]: I0708 08:37:18.233063    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hwn2m_09c5f560-1820-422f-823b-7c90342c0793/kube-storage-version-migrator-operator/0.log"